Collaborative Research: RI: Medium: Multilingual Long-form QA with Retrieval-Augmented Language Models

本项目旨在使自动问答系统能够生成段落级答案。以前的问答工作集中在可以用简短短语回答的简单问题上。构建生成段落级答案的系统为回答复杂问题提供了令人兴奋的机会，并为简单问题提供了更细致和全面的答案。该项目将为长式问答（LFQA）创建全面可靠的评估协议，开创多语言研究，以扩大更广泛人群的信息访问，并开发将网络搜索与LFQA系统集成的新算法，以提供可验证的长式答案与人类书面证据文件。该项目侧重于LFQA的三个核心维度-数据集，评估和建模。本研究将扩展以前以英语为中心的LFQA的范围，通过构建多语言LFQA数据集和研究跨语言的知识转移来研究大型语言模型的多语言能力。在建模方面，它将提出一个新的框架，迭代编织在一起-在一个透明的方式-从文档中检索的知识和记忆的知识从语言模型。最后，对于评估，该项目将聘请熟悉问题主题的领域专家，为他们对模型生成的答案的评估提供依据。这些反馈将被用于导出一个细粒度的注释框架，该框架定位错误并解开生成的答案的弱点。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to enable automatic question answering systems to produce paragraph-level answers. Prior work on question answering has focused on simpler questions that can be answered with short phrases. Building systems to produce paragraph-level answers opens up exciting opportunities to answer complicated questions, and to offer more nuanced and comprehensive answers to simpler questions. This project will create comprehensive and reliable evaluation protocols for long form question answering (LFQA), pioneer multilingual studies to broaden information access to a wider population, and develop new algorithms that integrate web search with LFQA systems to provide verifiable long form answers paired with human-written evidence documents. This project focuses on three core dimensions of LFQA – datasets, evaluation, and modeling. Expanding the scope of prior English-centric LFQA, this research will investigate multilingual capabilities of large language models by constructing multilingual LFQA datasets and studying knowledge transfer across languages. In terms of modeling, it will propose a new framework that iteratively weaves together – in a transparent manner—knowledge retrieved from documents and memorized knowledge from a language model. Finally for evaluation, the project will engage domain experts who are familiar with the question topic to provide rationales for their evaluation of model generated answers. Such feedback will be used to derive a fine-grained annotation framework which localizes errors and unpack the weaknesses of generated answers. Together, the proposed work will bring significant progress to LFQA, an emerging topic for natural language processing and artificial intelligence research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.