CAREER: Teaching Machines through Human Explanation for Information Extraction

职业：通过人类解释来教学机器以进行信息提取

• 批准号: 2048211
• 负责人: Xiang Ren
• 金额: $ 51.31万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-05-15 至 2026-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2048211&HistoricalAwards=false
• 关键词: CAREER; Teaching; Machines; through; Human

The majority of data being generated by society is free-form textual data. As the volume of text data continues to grow, humans alone cannot hope to be able to understand every piece of textual information published. Hence, a need for machine-based methods to extract salient entities along with their relationships from massive textual data is needed. While efforts to develop such methods have proven successful in academia, that success rarely translates over to practitioners employing extraction systems to solve real-world problems. A significant cause of this failure in translation is the requirement of copious amounts of training examples for a machine to learn extraction models. Even when a sufficient number of examples exist, machines learn very rigid methods, such that even a slight misspelling can cause a failure. Therefore, a re-think of how we develop, refine and maintain such machine-based extraction methods is required. This project proposes a new methodology based around the idea of providing explanations for both correct and incorrect decisions made by a machine, with the intention of requiring far fewer examples for machine training, as well as providing a process of softening the rigidity of current extraction methods. To achieve this goal, this project will solicit (from humans) natural language explanations on how a machine should reason about their task, as well as explanations correcting erroneous reasoning and an alerting system for when a machine’s rationale is possibly going wrong. Rather than treating humans as merely a “source of labels”, this project aims at developing a new learning framework that directly models a human’s natural language explanations to either provide a machine with labeling rationale or correct an observed erroneous rationale. The project will develop explanation-based learning methods that can capture the compositional nature of human natural language explanations, and study explanation-guided model refinement methods to update model parameters based on the provided human explanations regarding undesirable behaviors. To adapt to changing data distribution, this project will formulate a human-in-the-loop continual model refinement framework where problematic model behavioral patterns are automatically identified, and human feedback is solicited to correct the model. With these advancements, the project looks to fundamentally change the way models are trained, refined and updated, and look to do it by exploiting the expert knowledge contained within human explanations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

社会产生的大部分数据都是自由格式的文本数据。随着文本数据量的不断增长，人类无法单独理解发布的每一条文本信息。因此，需要基于机器的方法来从大量文本数据中提取显著实体沿着它们的关系。虽然开发这种方法的努力在学术界已被证明是成功的，但这种成功很少转化为从业者使用提取系统来解决现实世界的问题。翻译失败的一个重要原因是机器需要大量的训练示例来学习提取模型。即使有足够数量的例子，机器也会学习非常严格的方法，即使是轻微的拼写错误也会导致失败。因此，我们需要重新思考如何开发、改进和维护这种基于机器的提取方法。该项目提出了一种新的方法，其思想是为机器做出的正确和不正确的决定提供解释，目的是需要更少的机器训练示例，并提供一个软化当前提取方法刚性的过程。为了实现这一目标，该项目将（从人类那里）征求关于机器应该如何推理其任务的自然语言解释，以及纠正错误推理的解释和机器推理可能出错时的警报系统。该项目的目的不是将人类仅仅视为“标签来源”，而是开发一种新的学习框架，直接对人类的自然语言解释进行建模，为机器提供标记原理或纠正观察到的错误原理。该项目将开发基于解释的学习方法，可以捕捉人类自然语言解释的组成性质，并研究解释引导的模型细化方法，以根据提供的人类对不良行为的解释更新模型参数。为了适应不断变化的数据分布，该项目将制定一个人在回路的持续模型细化框架，其中自动识别有问题的模型行为模式，并征求人类反馈以纠正模型。随着这些进步，该项目希望从根本上改变模型的训练，改进和更新方式，并希望通过利用人类解释中包含的专家知识来实现这一目标。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。