CRII: CHS: Harnessing Machine Learning to Improve Human Decision Making: A Case Study on Deceptive Detection

CRII：CHS：利用机器学习改善人类决策：欺骗检测案例研究

• 批准号: 2125113
• 负责人: Chenhao Tan
• 金额: $ 17.5万
• 依托单位: University of Chicago
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-01-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2125113&HistoricalAwards=false
• 关键词: CRII; CHS; Harnessing; Machine; Learning

Humans are the final decision-makers in a wide variety of critical tasks that involve ethical and legal concerns, ranging from predicting criminal recidivism by the courts, to medical diagnosis, to identifying misleading information. These are challenging tasks for humans and for machines. However, for some closely-constrained tasks where vast amounts of training data are available, machine learning algorithms can outperform humans. If the knowledge encoded in the machine learning models can be elucidated to humans, these implementations can support human decision making and even tutor humans to achieve better performance. Those are the goals of this project.This project investigates human decision making with assistance from machine learning models for the task of detecting deception. It explores two domains routinely encountered on the Internet, online reviews and news articles. It develops two forms of assistance from machine learning models to improve human decision making while retaining human agency: 1) providing information based on machine learning models for real-time support of human decisions, and 2) automatically generating tutorials to help humans understand the nature of this task from the perspective of machine learning models (offline training). This project develops novel algorithms that incorporate educational psychology to help teach humans the knowledge encoded in machine learning algorithms. The project evaluates the two forms of assistance by tracking human performance improvement in user studies. The project explores additional indicators, such as trust and time to complete tasks, to further understand collaboration between humans and machine learning algorithms. The knowledge gained in the project will inform design principles for effective integration of artificial intelligence into human decision making.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在涉及伦理和法律的问题的各种关键任务中，人类是最终的决策者，从法院预测犯罪累犯，到医学诊断，再到识别误导性信息。这些对于人类和机器来说都是具有挑战性的任务。然而，对于一些有大量训练数据的严格约束的任务，机器学习算法可以胜过人类。 如果机器学习模型中编码的知识可以向人类阐明，这些实现可以支持人类决策，甚至指导人类实现更好的性能。这些就是这个项目的目标。这个项目在机器学习模型的帮助下研究人类的决策，以完成检测欺骗的任务。它探讨了两个领域经常遇到的互联网上，在线评论和新闻文章。它从机器学习模型中开发了两种形式的帮助，以改善人类决策，同时保留人类代理：1）提供基于机器学习模型的信息，以实时支持人类决策，以及2）自动生成教程，以帮助人类从机器学习模型的角度理解这项任务的性质（离线训练）。 该项目开发了新的算法，结合了教育心理学，以帮助人类学习机器学习算法中编码的知识。该项目通过在用户研究中跟踪人的绩效改进来评估这两种形式的援助。该项目探索了其他指标，如信任和完成任务的时间，以进一步了解人类和机器学习算法之间的协作。 该项目中获得的知识将为人工智能有效集成到人类决策中的设计原则提供信息。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making

• DOI: 10.1145/3479552
• 发表时间: 2021-01
• 期刊: Proceedings of the ACM on Human-Computer Interaction
• 作者: Han Liu;Vivian Lai;Chenhao Tan