EAGER: Integrating Dense Paraphrased-Enriched Representations with Large Language Models

目前的大型语言模型，如Chat-GPT，因其回答广泛的用户问题的能力而引起了人们的关注。但是这些模型仍然会被书面文本中没有特别提到的内容所混淆。例如，虽然食谱可能说“切洋葱”，但它并没有说最后会得到洋葱片或使用刀。正是语言的自然经济性使得当前的语言处理工具举步维艰。该项目开发了一种称为密集释义的方法，该方法通过释义来丰富表层语言，揭示文本中单词之间的深层关系，为当前模型提供所需的信息，以便从文本中进行更好的推理和判断。在该项目中，我们开发了一种基于意义如何与语言中的许多不同句法模式相关联的文本丰富模型Dense Paraphrasing。该项目开发了将密集释义与大型语言模型相结合的过程，然后对所产生的系统进行评估。这涉及四个子任务，在这里概述：（1）创建包含丰富注释的数据集，该注释使文本中隐藏的语义关系显式化;（2）通过微调大型语言模型，开发用于将源文本翻译为“密集”文本形式的密集释义文本生成系统的基线模型;（3）探讨密集释义在下游NLP任务中的应用，应用于问题分类任务;（4）应用于涉及对象在事件序列中移动或变化的共指消解问题。该项目的最终目标是构建一个新的框架，将先进的大型语言模型与更深层次的语义相结合，并将其集成到下游应用中，以推进NLP的最新技术水平，使系统更普遍地适用于现实世界的问题。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Current large language models like Chat-GPT have captured attention by their impressive ability to answer a broad range of user questions. But these models can still get confused by content not specifically mentioned in written text. For example, while a recipe may say “cut the onions”, it does not say that one ends up with onion pieces or that a knife was used. It is the natural economy of language that often makes life hard for current language processing tools. This project develops an approach called Dense Paraphrasing that enriches the surface language with paraphrases that reveal the deeper relations between words in a body of text, providing information needed by current models to make better inferences and judgments from the text.In this project, we develop a model, Dense Paraphrasing, for text enrichment based on how a meaning can be associated with many different syntactic patterns in language. This project develops the process of integrating Dense Paraphrasing with Large Language Models, followed by an evaluation of the resulting system. This involves four subtasks, outlined here: (1) creation of a dataset that contains a rich annotation that makes explicit the hidden semantic relations in the text; (2) the development of baseline models for Dense Paraphrasing text generation systems that translate the source text into the “dense” text form, by fine-tuning the Large Language Models; (3) applications of Dense Paraphrasing on downstream NLP tasks are explored, applied to the task of Question Answering; and (4) applied to a coreference resolution problem involving objects as they move or change through a sequence of events. The ultimate goal of the project is to build a novel framework incorporating advanced Large Language Models with deeper semantics and integrate it in downstream applications to advance the state of the art in NLP, making the systems more generally useful for real-world problems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.