EAGER: Integrating Dense Paraphrased-Enriched Representations with Large Language Models

当前的大型语言模型，如Chat-GPT，因其令人印象深刻的回答各种用户问题的能力而引起了人们的注意。但是这些模型仍然会被书面文本中没有特别提到的内容所混淆。例如，虽然一个食谱可能会说“切洋葱”，但它并没有说最后得到的是洋葱片，也没有说要用刀。语言的自然经济特性常常使当前的语言处理工具举步维艰。该项目开发了一种称为密集意译的方法，通过意译来丰富表面语言，揭示文本主体中单词之间更深层次的关系，提供当前模型所需的信息，以便从文本中做出更好的推断和判断。在这个项目中，我们开发了一个模型，密集释义，基于一个意义如何与语言中许多不同的句法模式相关联来丰富文本。该项目开发了将密集释义与大型语言模型集成的过程，随后对结果系统进行评估。这涉及四个子任务，在这里概述：(1)创建一个包含丰富注释的数据集，该注释使文本中隐藏的语义关系显化；(2)开发密集释义文本生成系统的基线模型，通过对大型语言模型进行微调，将源文本翻译成“密集”文本形式；(3)探讨了密集意译在下游NLP任务中的应用，并将其应用于问答任务；(4)应用于涉及物体通过一系列事件移动或变化的共参考解决问题。该项目的最终目标是建立一个新的框架，结合具有更深语义的高级大型语言模型，并将其集成到下游应用程序中，以推进NLP的最新技术，使系统对现实世界的问题更普遍有用。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Current large language models like Chat-GPT have captured attention by their impressive ability to answer a broad range of user questions. But these models can still get confused by content not specifically mentioned in written text. For example, while a recipe may say “cut the onions”, it does not say that one ends up with onion pieces or that a knife was used. It is the natural economy of language that often makes life hard for current language processing tools. This project develops an approach called Dense Paraphrasing that enriches the surface language with paraphrases that reveal the deeper relations between words in a body of text, providing information needed by current models to make better inferences and judgments from the text.In this project, we develop a model, Dense Paraphrasing, for text enrichment based on how a meaning can be associated with many different syntactic patterns in language. This project develops the process of integrating Dense Paraphrasing with Large Language Models, followed by an evaluation of the resulting system. This involves four subtasks, outlined here: (1) creation of a dataset that contains a rich annotation that makes explicit the hidden semantic relations in the text; (2) the development of baseline models for Dense Paraphrasing text generation systems that translate the source text into the “dense” text form, by fine-tuning the Large Language Models; (3) applications of Dense Paraphrasing on downstream NLP tasks are explored, applied to the task of Question Answering; and (4) applied to a coreference resolution problem involving objects as they move or change through a sequence of events. The ultimate goal of the project is to build a novel framework incorporating advanced Large Language Models with deeper semantics and integrate it in downstream applications to advance the state of the art in NLP, making the systems more generally useful for real-world problems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.