Cross-Task Learning for Natural Language Processing

该项目考虑了同时解决多组件级自然语言处理问题的问题。这样的组件级任务对于大规模应用程序(例如。如自动文档摘要，机器翻译等)，但通常是独立解决的。这些独立的解决方案忽略了将一个问题的输出与另一个问题的输出联系起来的自然联系。本研究探索了利用这种输出对应关系来辅助机器学习算法的能力，称为“跨任务学习”。这些输出对应关系提供了关于多个问题的期望输出之间关系的强先验信息。这种先验知识可以潜在地用于提高任务级性能，即使在大量训练数据不可用的情况下。该研究利用这种先验知识使用k-最佳方法，以最大限度地提高这些技术的适用性。它还开发了基于输出对应思想的半监督学习新技术，以便利用大量可用的未注释数据。此外，本文还在计算学习理论的背景下对所提出的技术进行了分析。其结果将是一套用于跨多个自然语言处理任务学习的技术。该技术将在低级任务（如浅解析和命名实体识别）以及高级任务（如话语分析和自动文档摘要）的背景下进行经验评估。

This project considers the problem of simultaneously solving multiplecomponent-level natural language processing problems. Suchcomponent-level tasks are necessary as building blocks for large-scaleapplications (eg., automatic document summarization, machinetranslation, etc.), but are typically solved independently. Theseindependent solutions ignore the natural connections that relate theoutput of one problem to the output of the other. This researchexplores the ability to exploit such output correspondences to aidmachine learning algorithms, termed "Cross-Task Learning." Theseoutput correspondences provide strong prior information about therelationship between the desired outputs of multiple problems. Thisprior knowledge can potentially serve to improve task-levelperformance, even when large amounts of training data are unavailable.The research exploits such prior knowledge using a k-best methodologyso as to maximize the applicability of these techniques. It alsodevelops new techniques for semi-supervised learning based on the ideaof output correspondences in order to capitalize on the vast amountsof unannotated data that are available. In addition, the proposed techniquesare analyzed in the context of computational learning theory. The outcome will be a set of techniques for learning across multiple natural language processingtasks. This technology will be empirically evaluated in the contextof low-level tasks such as shallow parsing and named entityrecognition, as well as the high-level tasks of discourse analysis andautomatic document summarization.