Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability
Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability
复制标题
DOI：
10.48550/arxiv.2206.08363
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Jonathan Crabbe;Alicia Curth;Ioana Bica;M. Schaar
中科院分区：
文献类型：
--
作者：
Jonathan Crabbe;Alicia Curth;Ioana Bica;M. Schaar
估计治疗的个性化效果是一个复杂但普遍的问题。为了解决这个问题，关于异质治疗效果估计的机器学习 (ML) 文献的最新发展催生了许多复杂但不透明的工具：由于其灵活性、模块化和学习约束表示的能力，神经网络尤其成为该文献的核心。不幸的是，此类黑匣子的资产是有代价的：模型通常涉及无数的重要操作，因此很难理解它们所学到的内容。然而，理解这些模型可能至关重要——例如，在医学背景下，发现的治疗效果异质性知识可以为临床实践中的治疗处方提供信息。因此，在这项工作中，我们使用事后特征重要性方法来识别影响模型预测的特征。这使我们能够沿着一个在以前的工作中被忽视的新的重要维度来评估治疗效果估计器：我们构建了一个基准环境来实证研究个性化治疗效果模型识别预测协变量的能力，即确定对治疗的差异反应的协变量。然后，我们的基准测试环境使我们能够在调整针对治疗效果估计的不同挑战时，对不同类型的治疗效果模型的优点和缺点提供新的见解——例如预后与预测信息的比率、潜在结果可能的非线性以及混杂因素的存在和类型。
Estimating personalized effects of treatments is a complex, yet pervasive problem. To tackle it, recent developments in the machine learning (ML) literature on heterogeneous treatment effect estimation gave rise to many sophisticated, but opaque, tools: due to their flexibility, modularity and ability to learn constrained representations, neural networks in particular have become central to this literature. Unfortunately, the assets of such black boxes come at a cost: models typically involve countless nontrivial operations, making it difficult to understand what they have learned. Yet, understanding these models can be crucial -- in a medical context, for example, discovered knowledge on treatment effect heterogeneity could inform treatment prescription in clinical practice. In this work, we therefore use post-hoc feature importance methods to identify features that influence the model's predictions. This allows us to evaluate treatment effect estimators along a new and important dimension that has been overlooked in previous work: We construct a benchmarking environment to empirically investigate the ability of personalized treatment effect models to identify predictive covariates -- covariates that determine differential responses to treatment. Our benchmarking environment then enables us to provide new insight into the strengths and weaknesses of different types of treatment effects models as we modulate different challenges specific to treatment effect estimation -- e.g. the ratio of prognostic to predictive information, the possible nonlinearity of potential outcomes and the presence and type of confounding.