Collaborative Research: RI: Medium: Multilingual Long-form QA with Retrieval-Augmented Language Models

该项目旨在使自动问答系统能够产生段落级别的答案。以前关于问题回答的工作集中在可以用简短短语回答的更简单的问题上。构建生成段落级答案的系统为回答复杂问题提供了令人兴奋的机会，并为更简单的问题提供了更细微和更全面的答案。该项目将为长格式问题回答(LFQA)创建全面和可靠的评估协议，开创多语言研究的先河，以扩大更广泛人群的信息获取渠道，并开发将网络搜索与LFQA系统相结合的新算法，以提供可核实的长格式答案与人类书面证据文件配对。本项目关注LFQA的三个核心维度--数据集、评估和建模。扩展以往以英语为中心的LFQA的范围，本研究将通过构建多语言LFQA数据集和研究跨语言的知识转移来考察大型语言模型的多语言能力。在建模方面，它将提出一个新的框架，以透明的方式将从文件中检索的知识和从语言模型中记忆的知识迭代地编织在一起。最后，对于评估，该项目将聘请熟悉问题主题的领域专家为他们评估模型生成的答案提供理由。这样的反馈将被用来派生一个细粒度的注释框架，该框架定位错误并解包生成的答案的弱点。总之，拟议的工作将为LFQA带来重大进展，LFQA是自然语言处理和人工智能研究的一个新兴课题。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to enable automatic question answering systems to produce paragraph-level answers. Prior work on question answering has focused on simpler questions that can be answered with short phrases. Building systems to produce paragraph-level answers opens up exciting opportunities to answer complicated questions, and to offer more nuanced and comprehensive answers to simpler questions. This project will create comprehensive and reliable evaluation protocols for long form question answering (LFQA), pioneer multilingual studies to broaden information access to a wider population, and develop new algorithms that integrate web search with LFQA systems to provide verifiable long form answers paired with human-written evidence documents. This project focuses on three core dimensions of LFQA – datasets, evaluation, and modeling. Expanding the scope of prior English-centric LFQA, this research will investigate multilingual capabilities of large language models by constructing multilingual LFQA datasets and studying knowledge transfer across languages. In terms of modeling, it will propose a new framework that iteratively weaves together – in a transparent manner—knowledge retrieved from documents and memorized knowledge from a language model. Finally for evaluation, the project will engage domain experts who are familiar with the question topic to provide rationales for their evaluation of model generated answers. Such feedback will be used to derive a fine-grained annotation framework which localizes errors and unpack the weaknesses of generated answers. Together, the proposed work will bring significant progress to LFQA, an emerging topic for natural language processing and artificial intelligence research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.