RI: Medium: Assessment of Machine Learning Algorithms in the Wild

机器学习现在是智能软件背后的一个关键方面，智能软件存在于我们日常生活的许多方面，包括摄像头中的人脸识别、可以回答问题的聊天机器人，以及我们手机上的语音识别和语言翻译。驱动这类软件的预测模型是通过基于大量历史数据训练的机器学习算法自动创建的。这些模型的一个问题是，它们往往是自然界中的黑匣子，人类很难理解。特别是，他们可能对自己的预测过于自信，并不总是能够认识到自己的局限性。随着这些类型的机器学习模型进入更关键的任务，如自动驾驶和医疗诊断，了解这些模型在现实世界实际情况中的局限性变得越来越重要。本研究项目将通过研究新的数学和算法方法来解决这些问题，这些方法可以提高我们评估黑盒预测模型的性能和置信度的能力，特别是当模型在以前从未遇到过的新环境中运行时。这项研究的结果将有可能显著提高机器学习系统在医疗、交通、商业和消费者应用等广泛领域的可靠性和可用性。本项目侧重于可解释人工智能的一个方面，涉及使黑盒机器学习模型(特别是基于分类和回归的模型)能够为其预测产生置信度声明。这个项目正在寻求新的方法来理解这些模型的预测，以克服隐含的过度自信，否则非黑即白、非或非的分类结果可能意味着。这一研究项目将在两大主题的背景下汇集认知科学和计算机科学的专业知识。第一个主题将专注于开发准确和健壮的算法，这些算法可以学习对黑盒模型的预测有多大的信心。研究人员将研究新的贝叶斯校准方法，并开发一个广泛的框架，用于对黑盒预测模型的能力进行稳健和准确的在线评估。第二个主题将利用第一个主题的算法进展，开发新的信心评估方法，以提高黑箱预测者和人类决策者共同努力的有效性。这项工作将反过来为权衡预测精度和人类努力提供基础，并允许开发利用准确的置信度估计来减少算法厌恶并增加人类部分信任的技术。吸引更广泛社区的兴趣也将是该项目的一个关键方面，重点是南加州代表性不足的社区大学生参加的研讨会和黑客松，以解决与人工智能和机器学习技术在我们的日常世界中使用相关的广泛问题。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning is now a key aspect behind the smart software that is present in many aspects of our daily lives, including face recognition in cameras, chatbots that can answer questions, and speech recognition and language translation on our mobile phones. The prediction models that drive such software are created automatically by machine learning algorithms trained on large amounts of historical data. One issue with these models is that they are often black-box in nature and difficult for humans to understand. In particular, they can be overconfident in their predictions and are not always able to recognize their own limitations. As these types of machine learning models move into more critical tasks, such as autonomous driving and medical diagnosis, it is becoming increasingly important to understand the limitations of such models in real-world practical situations. This research project will address these issues by investigating new mathematical and algorithmic approaches that can improve our ability to assess the performance and confidence of black-box prediction models, particularly when the models are operating in new environments that they have not encountered before. The outcomes of this research will have the potential to significantly improve the reliability and usability of machine learning systems across a broad range of areas such as medicine, transportation, business, and consumer applications.This project focuses on an aspect of explainable AI concerned with enabling black-box machine learning models (specifically those based on classification and regression) to produce confidence statements about their predictions. This project is pursuing novel methods for understanding the predictions of these models to overcome the implicit overconfidence that otherwise black-and-white, in-or-out classification outcomes can imply. This research project will bring together expertise from cognitive science and computer science in the context of two broad themes. The first theme will focus on developing accurate and robust algorithms that can learn how much confidence to place in a black-box model's predictions. The researchers will investigate new Bayesian calibration methods and develop a broad framework for robust and accurate online assessment of the capabilities of black-box prediction models. The second theme will leverage the algorithmic advances from the first theme to develop new approaches to confidence assessment that can improve the effectiveness of the combined efforts of a black-box predictor and a human decision-maker. This work will in turn provide the basis for trading off prediction accuracy and human effort, and allow for development of techniques that leverage accurate confidence estimates to reduce algorithm aversion and increase trust on the part of the human. Engaging the interest of a broader community will also be a key aspect of the project, with a focus on workshops and hackathons involving under-represented community college students in Southern California, to address broad-ranging questions related to the use of artificial intelligence and machine learning techniques in our everyday world.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.