Cross-Task Learning for Natural Language Processing

自然语言处理的跨任务学习

• 批准号: 0712764
• 负责人: Hal Daume
• 金额: $ 37.11万
• 依托单位: University of Utah
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2007
• 资助国家: 美国
• 起止时间: 2007-08-15 至 2011-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0712764&HistoricalAwards=false
• 关键词: Cross; Task; Learning; Natural; Language

This project considers the problem of simultaneously solving multiplecomponent-level natural language processing problems. Suchcomponent-level tasks are necessary as building blocks for large-scaleapplications (eg., automatic document summarization, machinetranslation, etc.), but are typically solved independently. Theseindependent solutions ignore the natural connections that relate theoutput of one problem to the output of the other. This researchexplores the ability to exploit such output correspondences to aidmachine learning algorithms, termed "Cross-Task Learning." Theseoutput correspondences provide strong prior information about therelationship between the desired outputs of multiple problems. Thisprior knowledge can potentially serve to improve task-levelperformance, even when large amounts of training data are unavailable.The research exploits such prior knowledge using a k-best methodologyso as to maximize the applicability of these techniques. It alsodevelops new techniques for semi-supervised learning based on the ideaof output correspondences in order to capitalize on the vast amountsof unannotated data that are available. In addition, the proposed techniquesare analyzed in the context of computational learning theory. The outcome will be a set of techniques for learning across multiple natural language processingtasks. This technology will be empirically evaluated in the contextof low-level tasks such as shallow parsing and named entityrecognition, as well as the high-level tasks of discourse analysis andautomatic document summarization.

该项目考虑同时解决多个分量级自然语言处理问题的问题。 这样的组件级任务对于大规模应用程序（例如，自动文档摘要、机器翻译等），但通常是独立求解的。 这些独立的解决方案忽略了一个问题的输出与另一个问题的输出之间的自然联系。 这项研究探索了利用这种输出对应来帮助机器学习算法的能力，称为“跨任务学习”。“这些输出对应关系提供了关于多个问题的期望输出之间关系的强先验信息。 这些先验知识可以潜在地提高任务级性能，即使在大量训练数据不可用的情况下，该研究使用k-best方法来利用这些先验知识，以最大限度地提高这些技术的适用性。 它还基于输出对应的思想开发了半监督学习的新技术，以便利用大量可用的未注释数据。此外，所提出的技术进行了分析的计算学习理论的背景下。其结果将是一套跨多个自然语言处理任务学习的技术。 该技术将在诸如浅层解析和命名实体识别等低级任务以及话语分析和自动文档摘要等高级任务的上下文中进行经验评估。