Mathematics > Optimization and Control
[Submitted on 1 Jul 2021 (v1), last revised 2 Aug 2021 (this version, v3)]
Title:About evaluation metrics for contextual uplift modeling
View PDFAbstract:In this tech report we discuss the evaluation problem of contextual uplift modeling from the causal inference point of view. More particularly, we instantiate the individual treatment effect (ITE) estimation, and its evaluation counterpart. First, we unify two well studied fields: the statistical ITE approach and its observational counterpart based on uplift study. Then we exhibit the problem of evaluation, based on previous work about ITE and uplift modeling. Moreover, we derive a new estimator for the uplift curve, built on logged bandit feedback dataset, that reduces its variance. We prove that AUUC fails on non randomized control trial (RCT) datasets and discuss some corrections and guidelines that should be kept in mind while using AUUC (re-balancing the population, local importance sampling using propensity score).
Submission history
From: Matthieu Martin [view email][v1] Thu, 1 Jul 2021 15:28:59 UTC (3,185 KB)
[v2] Fri, 2 Jul 2021 08:24:55 UTC (3,185 KB)
[v3] Mon, 2 Aug 2021 09:59:39 UTC (3,421 KB)
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.