Generative Language Models for Paragraph-Level Question Generation-猫眼课题宝

DOI：

10.48550/arxiv.2210.03992

发表时间：

2022-10

期刊：

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Asahi Ushio;Fernando Alva-Manchego;José Camacho-Collados

强大的生成模型导致了问题生成（QG）的最新进展。然而，很难衡量QG研究的进展，因为没有标准化的资源可以统一比较各种方法。在本文中，我们介绍了QG- bench，这是一个多语言和多领域的QG基准，通过将现有的问答数据集转换为标准的QG设置来统一现有的问答数据集。它包括通用数据集，如英语的SQuAD，来自10个领域和两种风格的数据集，以及8种不同语言的数据集。使用QG-Bench作为参考，我们对该任务的语言模型的功能进行了广泛的分析。首先，我们提出了基于微调生成语言模型的鲁棒QG基线。然后，我们用广泛的手动评估来补充基于标准度量的自动评估，这反过来揭示了评估QG模型的难度。最后，我们分析了这些模型的领域适应性以及多语言模型在非英语语言中的有效性。QG-Bench与论文中提出的微调模型一起发布（https://github.com/asahi417/lm-question-generation），也可以作为演示（https://autoqg.net/）。

Powerful generative models have led to recent progress in question generation (QG). However, it is difficult to measure advances in QG research since there are no standardized resources that allow a uniform comparison among approaches. In this paper, we introduce QG-Bench, a multilingual and multidomain benchmark for QG that unifies existing question answering datasets by converting them to a standard QG setting. It includes general-purpose datasets such as SQuAD for English, datasets from ten domains and two styles, as well as datasets in eight different languages. Using QG-Bench as a reference, we perform an extensive analysis of the capabilities of language models for the task. First, we propose robust QG baselines based on fine-tuning generative language models. Then, we complement automatic evaluation based on standard metrics with an extensive manual evaluation, which in turn sheds light on the difficulty of evaluating QG models. Finally, we analyse both the domain adaptability of these models as well as the effectiveness of multilingual models in languages other than English.QG-Bench is released along with the fine-tuned models presented in the paper (https://github.com/asahi417/lm-question-generation), which are also available as a demo (https://autoqg.net/).

Generative Language Models for Paragraph-Level Question Generation

文献摘要

相似文献