Career: Towards a Systematic Characterization of Model Explanations for High-Stakes Decision Making

随着机器学习(ML)模型越来越多地被用于在现实世界的应用程序中做出高风险的决策，确保相关的利益相关者能够理解和信任这些模型的功能变得至关重要。然而，ML模型日益增长的复杂性和专有性质使得利益相关者很难理解这些模型的行为。因此，近年来已经提出了几种方法来以人类可解释的方式解释ML模型的行为。然而，这些方法采用了截然不同的策略来解释模范行为，而且往往相互矛盾。这些解释方法的日益多样化，加上缺乏系统的评估框架，使得无法确定哪些方法可能在不同类型的关键现实世界应用程序中有效。这个项目将建立严格的框架，系统地分析、评估和比较各种最先进的解释方法在不同现实世界应用程序中的可靠性和实用性。作为该项目的一部分开发的框架有可能显著加快ML模型在各种环境中的采用，包括医疗保健(例如，患者治疗建议)、贷款(例如，贷款批准决定)和招聘(例如，简历筛选)。这个项目的目的是系统地描述现有的解释方法，以便从业者能够容易地确定在给定的现实世界应用程序中使用哪些方法。该项目将集中于以下子任务：1)开发新的理论和经验框架，以分析各种最先进的方法如何可靠地解释不同类型的ML模型(例如，线性模型与非线性模型)的行为；2)与医疗保健、借贷和招聘领域的专家一起进行大规模用户研究，以评估现有解释方法在不同高风险应用程序中的效用；3)利用从上述用户研究中获得的数据来构建能够模拟领域专家的行为的算法代理，并利用这些代理进而在规模上评估解释方法的效用，以及4)开发新的算法框架，可以根据给定的现实世界背景自动选择合适的解释方法。有了这些贡献，这项研究将为更清晰的理解和更广泛的共识铺平道路，关于哪些解释方法可能有效地解释各种高风险应用程序中不同ML模型的行为。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

As machine learning (ML) models are increasingly employed to make high-stakes decisions in real-world applications, it becomes crucial to ensure that the relevant stakeholders can understand and trust the functionality of these models. However, the increasing complexity and the proprietary nature of ML models make it rather challenging for stakeholders to understand the behavior of these models. Consequently, several methods have been proposed in recent years to explain the behavior of ML models in a human-interpretable fashion. These methods, however, adopt vastly different strategies to explain model behavior and often contradict each other. The increasing diversity of these explanation methods, coupled with the lack of systematic evaluation frameworks, have made it impossible to determine which methods are likely to be effective across different kinds of critical real-world applications. This project will build rigorous frameworks for systematically analyzing, evaluating, and comparing the reliability and utility of various state-of-the-art explanation methods across different real-world applications. The frameworks developed as part of this project have the potential to significantly accelerate the adoption of ML models in a variety of settings, including healthcare (e.g., patient treatment recommendations), lending (e.g., loan approval decisions), and hiring (e.g., resume screening). This project aims to systematically characterize existing explanation methods so that practitioners can readily determine which methods to employ in a given real-world application. The project will focus on the following subtasks: 1) developing novel theoretical and empirical frameworks to analyze how reliably various state-of-the-art methods explain the behavior of different types of ML models (e.g., linear vs. non-linear models), 2) conducting large-scale user studies with domain experts in healthcare, lending, and hiring to evaluate the utility of existing explanation methods across different high-stakes applications, 3) leveraging the data obtained from the aforementioned user studies to build algorithmic agents which can mimic the behavior of domain experts, and employing these agents to, in turn, evaluate the utility of explanation methods at scale, and 4) developing novel algorithmic frameworks which can automatically select an appropriate explanation method tailored to a given real-world context. With these contributions, this research will pave the way for a clearer understanding and a broader consensus on which explanation methods are likely to effectively explain the behavior of different ML models across various high-stakes applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.