Collaborative Research: RI: Medium: Multilingual Long-form QA with Retrieval-Augmented Language Models

该项目旨在使自动问答系统产生段落级答案。之前关于问题回答的工作集中在可以用短句回答的简单问题上。构建生成段落级答案的系统为回答复杂问题提供了令人兴奋的机会，并为更简单的问题提供了更细致和全面的答案。该项目将为长形式问答（LFQA）创建全面可靠的评估协议，开拓多语言研究，扩大信息获取范围，并开发新的算法，将网络搜索与长形式问答系统集成在一起，提供可验证的长形式答案，并与人类编写的证据文件相匹配。本项目聚焦于LFQA的三个核心维度——数据集、评估和建模。本研究将扩展先前以英语为中心的LFQA的范围，通过构建多语言LFQA数据集和研究跨语言的知识迁移来研究大型语言模型的多语言能力。在建模方面，它将提出一个新的框架，以一种透明的方式将从文档中检索到的知识和从语言模型中记忆的知识迭代地编织在一起。最后，为了评估，项目将聘请熟悉问题主题的领域专家，为他们对模型生成的答案的评估提供依据。这样的反馈将用于派生一个细粒度的注释框架，该框架可以定位错误并解压缩生成答案的弱点。总之，拟议的工作将为LFQA这一自然语言处理和人工智能研究的新兴课题带来重大进展。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to enable automatic question answering systems to produce paragraph-level answers. Prior work on question answering has focused on simpler questions that can be answered with short phrases. Building systems to produce paragraph-level answers opens up exciting opportunities to answer complicated questions, and to offer more nuanced and comprehensive answers to simpler questions. This project will create comprehensive and reliable evaluation protocols for long form question answering (LFQA), pioneer multilingual studies to broaden information access to a wider population, and develop new algorithms that integrate web search with LFQA systems to provide verifiable long form answers paired with human-written evidence documents. This project focuses on three core dimensions of LFQA – datasets, evaluation, and modeling. Expanding the scope of prior English-centric LFQA, this research will investigate multilingual capabilities of large language models by constructing multilingual LFQA datasets and studying knowledge transfer across languages. In terms of modeling, it will propose a new framework that iteratively weaves together – in a transparent manner—knowledge retrieved from documents and memorized knowledge from a language model. Finally for evaluation, the project will engage domain experts who are familiar with the question topic to provide rationales for their evaluation of model generated answers. Such feedback will be used to derive a fine-grained annotation framework which localizes errors and unpack the weaknesses of generated answers. Together, the proposed work will bring significant progress to LFQA, an emerging topic for natural language processing and artificial intelligence research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.