CAREER: Faithful Natural Language Generation

职业：忠实的自然语言生成

• 批准号: 2048122
• 负责人: William Wang
• 金额: $ 54.99万
• 依托单位: University of California-Santa Barbara
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-04-01 至 2026-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2048122&HistoricalAwards=false
• 关键词: CAREER; Faithful; Natural; Language; Generation

Natural Language Generation is a fundamental component of many real-world applications, including generating captions for visually-impaired people. The CAREER project will seek to develop new methods for generating higher quality and more reliable sentences, and ensure better and more faithful generation results. The project will also lead to open-source software and tools that facilitate the diagnosis of neural generation models, and provide resources for building the next generation faithful language generation models. The investigator will integrate research with educational components, and enable underrepresented high school students to access Artificial Intelligence and Natural Language Processing research and course materials. A major challenge that prevents deep learning based natural language generation models in practical deployment is faithfulness. For example, in the task of image captioning, when using sequence-to-sequence models for generation, it often leads to the “hallucination” phenomenon: an object that does not belong to the context might be generated in the text. Similarly, in the task of data-to-text generation (e.g., generating a Wikipedia biography from structured data) problem, deep learning models are prone to generate erroneous entities and attributes that do not belong to the input data. These behaviors significantly downgrade the performance of neural generative models, and the faithfulness of the output becomes a significant issue for building the next generation faithful natural language generation engines. This project will investigate the complex relationships between uncertainty and faithfulness at various levels. And several mitigation strategies will also be considered. An interactive agent will be built to reason in user-generated text to understand the faithfulness constraint. The goal of this project is to deeply understand how to quantify and access faithfulness in robust settings, and build useful open-source software that facilitates this purpose.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

自然语言生成是许多现实世界应用程序的基本组成部分，包括为视障人士生成字幕。CAREER项目将寻求开发新的方法来生成更高质量和更可靠的句子，并确保更好和更忠实的生成结果。该项目还将带来开源软件和工具，以促进神经生成模型的诊断，并为构建下一代忠实的语言生成模型提供资源。研究人员将把研究与教育部分结合起来，使代表性不足的高中生能够访问人工智能和自然语言处理研究和课程材料。在实际部署中，阻止基于深度学习的自然语言生成模型的一个主要挑战是忠实性。例如，在图像字幕的任务中，当使用序列到序列模型进行生成时，常常会导致“幻觉”现象：可能在文本中生成不属于上下文的对象。类似地，在数据到文本生成的任务中（例如，从结构化数据生成维基百科传记）问题，深度学习模型容易生成不属于输入数据的错误实体和属性。这些行为显著降低了神经生成模型的性能，并且输出的忠实性成为构建下一代忠实自然语言生成引擎的重要问题。这个项目将调查不确定性和忠诚之间的复杂关系在各个层面上。还将考虑几种缓解策略。一个交互式代理将建立在用户生成的文本中的原因，以了解忠诚度约束。该项目的目标是深入了解如何在稳健的环境中量化和获取忠诚度，并构建有助于实现这一目的的有用的开源软件。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值进行评估，被认为值得支持和更广泛的影响审查标准。

项目成果

Self-Supervised Knowledge Assimilation for Expert-Layman Text Style Transfer

• DOI: 10.1609/aaai.v36i10.21410
• 发表时间: 2021-10
• 作者: Wenda Xu;Michael Stephen Saxon;Misha Sra;W. Wang

ImaginE: An Imagination-Based Automatic Evaluation Metric for Natural Language Generation

• DOI: 10.18653/v1/2023.findings-eacl.6
• 发表时间: 2021-06
• 期刊: ArXiv
• 作者: Wanrong Zhu;X. Wang;An Yan;M. Eckstein;W. Wang

Mitigating Covertly Unsafe Text within Natural Language Systems

• DOI: 10.48550/arxiv.2210.09306
• 发表时间: 2022-10
• 作者: Alex Mei;Anisha Kabir;Sharon Levy;Melanie Subbiah;Emily Allaway;J. Judge;D. Patton;Bruce Bimber;K. McKeown;William Yang Wang

LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation

• DOI: 10.48550/arxiv.2305.11116
• 发表时间: 2023-05
• 期刊: ArXiv
• 作者: Yujie Lu;Xianjun Yang;Xiujun Li;X. Wang;William Yang Wang

Improving Few-Shot Generalization by Exploring and Exploiting Auxiliary Data

• DOI: 10.48550/arxiv.2302.00674
• 发表时间: 2023-02
• 期刊: ArXiv
• 作者: Alon Albalak;Colin Raffel;William Yang Wang