CAREER: Active Learning through Rich and Transparent Interactions

机器学习模型是在由人类注释（标记）的数据上训练的。 训练模型的准确性通常随着注释数据示例的数量而提高。然而，注释需要时间、金钱和精力。 主动学习的目的是通过确定哪些模板信息量最大，并将人类标注者引导到这些模板来最大限度地降低成本。 主动学习的改进将降低与数据注释相关的成本，并导致更快地实现智能系统，用于一系列应用，包括机器人技术，语音技术，错误和异常检测（例如在医学、金融欺诈和基于条件的基础设施维护方面）、定向广告、人机界面和生物信息学。在传统的主动学习方法中，算法在它们能够获取的信息类型方面受到限制，并且它们通常不向用户提供关于为什么选择特定样本进行注释的任何基本原理。 这个CAREER项目开发了一个新的范式，称为“丰富和透明的主动学习”。“这种新的范式打开了算法和用户之间的沟通渠道，他们可以交换丰富的查询，答案和解释。 通过使用来自用户的丰富反馈，算法将能够更经济地学习目标概念，减少构建准确预测模型所需的资源。 通过解释他们的推理，这些算法将实现透明度，建立信任，并接受审查。 为此，该项目开发了一些方法，允许算法使用丰富的查询集进行资源有效的模型训练，并生成信息丰富但不会让用户不知所措的解释。 开发的方法建立在预期损失最小化，信息论和人机交互的原则。 使用公开的数据集和作为项目一部分进行的用户研究对各种方法进行了评价。 该项目开发了两个高影响力的现实世界问题的案例研究：检测欺诈性医疗保健索赔，并识别处于疾病风险中的患者。丰富而透明的主动学习范式提供了独特的教育机会。 与黑箱操作的标准机器学习算法相比，交互式和透明的机器学习有望提高学生对数据科学的兴趣和动力。 两名博士和几名本科生和高中生正在接受该奖项的培训。 目前正在开发一门关于交互式机器学习的新的研究生课程。 最后，PI通过与芝加哥一所公立高中合作，确保有效地接触代表性不足的群体，该高中的学生人口中有90%是少数民族。

Machine learning models are trained on data that are annotated (labeled) by humans. The accuracy of the trained models generally improves with the number of annotated data examples. Yet, annotating takes time, money, and effort. Active learning aims to minimize the costs by determining which exemples are most informative and directing the human labeler to them. Improvements in active learning will lower the costs associated with data annotation and lead to faster implementations of intelligent systems for a range of applications including robotics, speech technology, error and anomaly detection (for example in medicine, financial fraud, and condition-based maintenance of infrastructure), targeted advertising, human-computer interfaces, and bioinformatics.In traditional active learning approaches, algorithms are limited in the types of information they can acquire, and they often do not provide any rationale to the user as to why a particular exemplar is chosen for annotation. This CAREER project develops a new paradigm dubbed "rich and transparent active learning." This new paradigm opens a communication channel between algorithms and users whereby they can exchange a rich set of queries, answers, and explanations. By using rich feedback from users the algorithms will be able to learn the target concept more economically, reducing the resources required to build an accurate predictive model. By explaining their reasoning, these algorithms will achieve transparency, build trust, and open themselves to scrutiny. Towards that end, the project develops methods that allow algorithms to use a rich set of queries for resource-efficient model training, and generate explanations that are informative but not overwhelming for the users. The methods developed build on expected loss minimization, information theory, and principles from human-computer interaction. Approaches are evaluated using publicly available datasets and user studies carried out as part of the project. The project develops case studies on two high-impact real-world problems: detecting fraudulent health-care claims, and identifying patients at risk of disease.The rich and transparent active learning paradigm provides unique educational opportunities. In contrast to standard machine learning algorithms, operated as black boxes, interactive and transparent machine learning is expected to raise students' interest and motivation for data science. Two PhD and several undergraduate and high school students are being trained under this award. A new graduate course on interactive machine learning is being developed. Finally the PI ensures effective outreach to under-represented groups by partnering with a Chicago public high school whose student population includes 90% minorities.