RI: Medium: Quantifying and utilizing confidence in machine learning

RI：中：量化和利用机器学习的信心

• 批准号: 1162581
• 负责人: Yoav Freund
• 金额: $ 100万
• 依托单位: University of California-San Diego
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-09-01 至 2017-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1162581&HistoricalAwards=false
• 关键词: RI; Medium; Quantifying; utilizing; confidence

This project defines meaningful notions of confidence in prediction, designs procedures for computing such notions, and applies these procedures to core machine learning tasks such as active learning, crowd-sourced learning, and tracking. In many applications it is helpful to have classifiers that output, together with each prediction, a rating of the confidence that the prediction is in fact correct. Existing literature either provides various ad-hoc ways for computing such ratings which typically lack a rigorous mathematical footing, or provides mathematically consistent methods (in the Bayesian framework) for computing confidence ratings under very strong assumptions that are unlikely to hold in practice. The research team investigates methods of computing measures of confidence that are mathematically rigorous while making minimal assumptions on the way data is generated, and use these measures to further develop solutions to core machine learning tasks.Defining and computing mathematically sound measures of confidence lies at the heart of machine learning, pattern recognition and uncertainty in AI. Confidence-rated prediction, active learning, and tracking are fundamental tasks of machine learning and statistics that arise repeatedly in large-scale problems; this project will develop rigorous solutions to these problems. The algorithms developed in this work are tested and used in the Automatic Cameraman project, an interactive, audio-visual installation in the UCSD Computer Science department. The interactive Automatic Cameraman system are used an educational tool to be extended in many different directions, by teams of students at a variety of skill levels.

该项目定义了有意义的预测信心概念，设计了计算这些概念的程序，并将这些程序应用于核心机器学习任务，如主动学习、众包学习和跟踪。在许多应用程序中，使用分类器输出预测实际上正确的置信度评级是很有帮助的。现有文献要么提供各种特别的方法来计算这种评级，通常缺乏严格的数学基础，要么提供数学上一致的方法（在贝叶斯框架中）在非常强的假设下计算置信度评级，这些假设在实践中不太可能成立。研究小组研究了计算置信度的方法，这些方法在数学上是严格的，同时对数据生成方式做出最小的假设，并使用这些措施进一步开发核心机器学习任务的解决方案。定义和计算在数学上合理的信心度量是人工智能中机器学习、模式识别和不确定性的核心。置信度预测、主动学习和跟踪是机器学习和统计学的基本任务，在大规模问题中反复出现；这个项目将为这些问题制定严格的解决方案。在这项工作中开发的算法被测试并用于自动摄像机项目，这是加州大学圣地亚哥分校计算机科学系的一个交互式视听装置。交互式自动摄像师系统是一种教育工具，可以扩展到许多不同的方向，由不同技能水平的学生组成。