III: SMALL: Moving Beyond Knowledge to Action: Evaluating and Improving the Utility of Causal Inference

机器学习最新的关键进展之一是能够从观测数据中学习因果结构。与相关性不同，原因让我们有力地预测未来，并确定哪些变量需要干预以潜在地改变它。因此，许多计算方法被引入，以更好地从日益可用的大型数据集中发现原因。然而，寻找原因的算法主要是根据它们恢复基本事实的准确性来评估的。这假设最完整和最准确的因果模型将是最有用的，但这一假设尚未得到检验，人们往往难以理解复杂的信息。因果模型也可以用来更好地理解行动的影响，这可能会进一步改善决策。虽然目前的方法确定了打开或关闭变量的影响，但对于做出选择的个人来说，这并不是正确的详细程度，例如糖尿病患者决定早餐吃什么特定的食物。此外，因果推理输出的使用者不是那些开发方法的人，而是具有不同水平的背景知识和感知的专业知识的人。这个项目的重点是通过量化因果模型的效用来缩小机器学习和人类决策之间的差距，引入使因果模型更有用和可用的新方法，并利用结果来改善围绕饮食和锻炼的日常决策。该项目旨在通过更好的因果推理评估指标和使因果模型更有用和更个性化的算法，关闭从数据到知识到行动的循环。这项工作将提高我们有效利用机器学习输出的能力，并鼓励开发产生高实用性输出的方法。首先，这个项目开发了一种新的方法来自动评估一组推断原因的效用，这使得算法可以沿着这个新的维度进行比较，从而提供对现实世界使用的更多洞察。具体地说，开发了考虑模型、用户和上下文特征的新度量，以允许对因果模型进行自动评分，以确定它们对决策的有用程度。其次，开发的指标被用来指导开发更有用的模型，这些模型准确地预测干预的效果，并纳入机械信息。将因果模型转化为现实世界使用的一个关键差距是需要预测干预措施的结果，这些干预措施可能不直接与变量相对应(例如，喝橙汁不同于直接增加血糖)。开发的新方法可以使用模拟来预测干预效果，并将模型映射到机械信息以实现进一步的洞察。最后，该项目证明了这些增强的因果模型可以改善现实生活中的决策。该项目可以帮助使机器学习的输出具有可操作性，并可能在许多与健康、金融和个人交通相关的重要决策场景中应用。这项研究可能会更普遍地改善决策，并可应用于减少分心驾驶和了解选择对能源使用的影响等不同领域。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

One of the key recent advances in machine learning is the ability to learn causal structures from observational data. Unlike correlations, causes let us robustly predict the future and identify which variables to intervene on to potentially change it. As a result, many computational methods have been introduced to better discover causes from the large datasets that are increasingly becoming available. However, algorithms for finding causes are mainly evaluated on how accurately they can recover ground truth. This assumes that the most complete and accurate causal model will be the most useful one, but this assumption has not been tested and people often struggle to make sense of complex information. Causal models can also be used to better understand the effects of actions, which could further improve decisions. While current methods identify the effects of turning a variable on or off, this is not the right level of detail for an individual making choices such as a person with diabetes deciding what specific food to consume for breakfast. Further, the users of the output of causal inference are not those developing the methods, but rather people with varying levels of background knowledge and perceived expertise. This project focuses on reducing the gap between machine learning and human decision-making by quantifying the utility of causal models, introducing new methods that make causal models more useful and usable, and leveraging the results to improve everyday decisions around diet and exercise. This project aims to close the loop from data to knowledge to action, through better metrics for evaluating causal inference, and algorithms that make causal models more useful and personalized. This work will advance our ability to effectively use the output of machine learning, and encourage the development of methods that produce output with high utility. First, this project develops novel ways to automatically evaluate the utility of a set of inferred causes, which allow algorithms to be compared along this new dimension that provides more insight into real-world use. In particular, new metrics are developed that take into account model, user, and context features to allow causal models to be automatically scored on how useful they are for decision-making. Second, the developed metrics are used to guide development of more useful models that accurately predict the effects of interventions and incorporate mechanistic information. A key gap translating causal models to real-world use is the need to predict the result of interventions that may not directly map to variables (e.g. drinking orange juice is not the same as directly increasing glucose). The new methods developed can predict intervention effects using simulation, and map models to mechanistic information to enable further insight. Lastly, the project demonstrates that these enhanced causal models can improve real-life decisions. The project can help make the output of machine learning actionable, and may have applications in many important decision-making scenarios related to health, finance, and personal transportation. The research may more generally improve decision-making, and can be applied to areas as diverse as reducing distracted driving and understanding the impact of choices on energy usage.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.