Retrieval-based Language Models and Applications
Retrieval-based Language Models and Applications
复制标题
DOI：
10.18653/v1/2023.acl-tutorials.6
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Akari Asai;Sewon Min;Zexuan Zhong;Danqi Chen
中科院分区：
文献类型：
--
作者：
Akari Asai;Sewon Min;Zexuan Zhong;Danqi Chen
基于检索的语言模型（LMS）在不同的NLP任务上表现出了令人印象深刻的性能。在本教程中，我们将对基于检索的LMS的最新进展提供全面而连贯的概述。首先，我们将提供涵盖LMS基础（例如蒙面LMS，自动回收LMS）和检索系统（例如，最近的邻居搜索）的初步。然后，我们将详细介绍基于检索的模型的最新进展，重点介绍其模型架构和学习方法。最后，我们将展示如何将基于检索的LMS适应下游应用程序，并扩展到多语言和多模式设置。最后，我们将使用练习来展示基于检索的LMS的有效性。
Retrieval-based language models (LMs) have shown impressive performance on diverse NLP tasks. In this tutorial, we will provide a comprehensive and coherent overview of recent advances in retrieval-based LMs. We will start by providing preliminaries covering the foundation of LMs (e.g., masked LMs, autoregressive LMs) and retrieval systems (e.g., nearest-neighbor search). We will then detail recent progress in retrieval-based models, focusing on their model architectures and learning approaches. Finally, we will show how retrieval-based LMs are adapted to downstream applications, and extended to multilingual and multi-modal settings. Finally, we will use an exercise to showcase the effectiveness of retrieval-based LMs.