III: SMALL: Moving Beyond Knowledge to Action: Evaluating and Improving the Utility of Causal Inference

机器学习最近的关键进展之一是从观测数据中学习因果结构的能力。与相关性不同，原因让我们能够有力地预测未来，并确定哪些变量可以干预以潜在地改变未来。因此，许多计算方法被引入，以便更好地从日益可用的大型数据集中发现原因。然而，寻找原因的算法主要是根据它们能多准确地恢复地面真相来评估的。这种假设认为，最完整、最准确的因果模型将是最有用的模型，但这种假设尚未经过检验，人们往往难以理解复杂的信息。因果模型也可以用来更好地理解行为的影响，这可以进一步改善决策。虽然目前的方法确定了打开或关闭变量的影响，但对于个人做出选择（例如糖尿病患者决定早餐吃什么特定的食物）来说，这并不是正确的细节水平。此外，因果推理输出的用户不是那些开发方法的人，而是具有不同水平的背景知识和感知专业知识的人。该项目专注于通过量化因果模型的效用，引入使因果模型更有用和可用的新方法，并利用结果来改善饮食和锻炼方面的日常决策，从而缩小机器学习与人类决策之间的差距。该项目旨在通过更好的评估因果推理的指标，以及使因果模型更有用和个性化的算法，实现从数据到知识到行动的闭环。这项工作将提高我们有效使用机器学习输出的能力，并鼓励开发产生高效用输出的方法。首先，该项目开发了新的方法来自动评估一组推断原因的效用，这使得算法可以沿着这个新的维度进行比较，从而更深入地了解现实世界的使用情况。特别是，开发了考虑模型、用户和上下文特征的新度量，以允许根据因果模型对决策的有用程度自动对其进行评分。其次，开发的指标用于指导开发更有用的模型，以准确预测干预措施的效果并纳入机制信息。将因果模型转化为实际应用的一个关键差距是，需要预测可能无法直接映射到变量的干预措施的结果（例如，饮用橙汁与直接增加葡萄糖不同）。开发的新方法可以通过模拟来预测干预效果，并将模型映射到机制信息，从而进一步了解。最后，该项目表明，这些增强的因果模型可以改善现实生活中的决策。该项目有助于使机器学习的输出具有可操作性，并可能在与健康、金融和个人交通相关的许多重要决策场景中得到应用。这项研究可能更广泛地改善决策，并可应用于各种领域，如减少分心驾驶和了解选择对能源使用的影响。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

One of the key recent advances in machine learning is the ability to learn causal structures from observational data. Unlike correlations, causes let us robustly predict the future and identify which variables to intervene on to potentially change it. As a result, many computational methods have been introduced to better discover causes from the large datasets that are increasingly becoming available. However, algorithms for finding causes are mainly evaluated on how accurately they can recover ground truth. This assumes that the most complete and accurate causal model will be the most useful one, but this assumption has not been tested and people often struggle to make sense of complex information. Causal models can also be used to better understand the effects of actions, which could further improve decisions. While current methods identify the effects of turning a variable on or off, this is not the right level of detail for an individual making choices such as a person with diabetes deciding what specific food to consume for breakfast. Further, the users of the output of causal inference are not those developing the methods, but rather people with varying levels of background knowledge and perceived expertise. This project focuses on reducing the gap between machine learning and human decision-making by quantifying the utility of causal models, introducing new methods that make causal models more useful and usable, and leveraging the results to improve everyday decisions around diet and exercise. This project aims to close the loop from data to knowledge to action, through better metrics for evaluating causal inference, and algorithms that make causal models more useful and personalized. This work will advance our ability to effectively use the output of machine learning, and encourage the development of methods that produce output with high utility. First, this project develops novel ways to automatically evaluate the utility of a set of inferred causes, which allow algorithms to be compared along this new dimension that provides more insight into real-world use. In particular, new metrics are developed that take into account model, user, and context features to allow causal models to be automatically scored on how useful they are for decision-making. Second, the developed metrics are used to guide development of more useful models that accurately predict the effects of interventions and incorporate mechanistic information. A key gap translating causal models to real-world use is the need to predict the result of interventions that may not directly map to variables (e.g. drinking orange juice is not the same as directly increasing glucose). The new methods developed can predict intervention effects using simulation, and map models to mechanistic information to enable further insight. Lastly, the project demonstrates that these enhanced causal models can improve real-life decisions. The project can help make the output of machine learning actionable, and may have applications in many important decision-making scenarios related to health, finance, and personal transportation. The research may more generally improve decision-making, and can be applied to areas as diverse as reducing distracted driving and understanding the impact of choices on energy usage.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.