Cross-Task Learning for Natural Language Processing

该项目考虑同时解决多构件级自然语言处理问题。对于大规模应用程序(例如，自动文档摘要、机器翻译等)来说，这些组件级别的任务是必需的，但通常是独立解决的。这些独立的解决方案忽略了将一个问题的输出与另一个问题的输出联系起来的自然联系。这项研究探索了利用这种输出对应来辅助机器学习算法的能力，这种算法被称为“跨任务学习”。这些输出对应关系提供了关于多个问题的期望输出之间的关系的强先验信息。即使在大量训练数据不可用的情况下，这种先验知识也可能有助于提高任务级的性能。本研究使用k-Best方法来利用这种先验知识，以最大限度地提高这些技术的适用性。它还开发了基于输出对应的思想的半监督学习的新技术，以便利用可用的大量未注释数据。此外，在计算学习理论的背景下对所提出的技术进行了分析。其结果将是一套跨多个自然语言处理任务进行学习的技术。该技术将在诸如浅层句法分析和命名实体识别等低级任务以及语篇分析和自动文档摘要的高级任务的背景下进行经验评估。

This project considers the problem of simultaneously solving multiplecomponent-level natural language processing problems. Suchcomponent-level tasks are necessary as building blocks for large-scaleapplications (eg., automatic document summarization, machinetranslation, etc.), but are typically solved independently. Theseindependent solutions ignore the natural connections that relate theoutput of one problem to the output of the other. This researchexplores the ability to exploit such output correspondences to aidmachine learning algorithms, termed "Cross-Task Learning." Theseoutput correspondences provide strong prior information about therelationship between the desired outputs of multiple problems. Thisprior knowledge can potentially serve to improve task-levelperformance, even when large amounts of training data are unavailable.The research exploits such prior knowledge using a k-best methodologyso as to maximize the applicability of these techniques. It alsodevelops new techniques for semi-supervised learning based on the ideaof output correspondences in order to capitalize on the vast amountsof unannotated data that are available. In addition, the proposed techniquesare analyzed in the context of computational learning theory. The outcome will be a set of techniques for learning across multiple natural language processingtasks. This technology will be empirically evaluated in the contextof low-level tasks such as shallow parsing and named entityrecognition, as well as the high-level tasks of discourse analysis andautomatic document summarization.