InSCIt : Information-Seeking Conversations with Mixed-Initiative Interactions
InSCIt : Information-Seeking Conversations with Mixed-Initiative Interactions
复制标题
InSCIt：具有混合主动交互的信息寻求对话
DOI：
10.1162/tacl_a_00559
复制
发表时间：
2023
影响因子：
10.9
通讯作者：
Hajishirzi, Hannaneh
中科院分区：
文献类型：
--
作者：
Wu, Zeqiu;Parish, Ryu;Cheng, Hao;Min, Sewon;Ammanabrolu, Prithviraj;Ostendorf, Mari;Hajishirzi, Hannaneh
在寻求信息的对话中，用户可能会问一些不明确或无法回答的问题。理想的代理应该通过根据可用的知识源发起不同的响应类型来进行交互。然而，目前的大多数研究要么未能或人为地纳入这种代理方的倡议。这项工作提出了InSCIt，一个具有混合主动交互的Information-SeekingConversations数据集。它包含来自805个人与人的对话的4.7K个用户-代理会话，其中代理在维基百科上搜索，并直接回答、要求澄清或提供相关信息来解决用户的问题。数据支持两个子任务，证据通道识别和响应生成，以及评估模型性能的人工评估协议。我们报告了基于最新的会话知识识别和开放领域问答模型的两个系统的结果。这两个系统的表现都明显逊于人类，这表明在未来的研究中有很大的改进空间。
In an information-seeking conversation, a user may ask questions that are under-specified or unanswerable. An ideal agent would interact by initiating different response types according to the available knowledge sources. However, most current studies either fail to or artificially incorporate such agent-side initiative. This work presentsInSCIt, a dataset forInformation-SeekingConversations with mixed-initiativeInteractions. It contains 4.7K user-agent turns from 805 human-human conversations where the agent searches over Wikipedia and either directly answers, asks for clarification, or provides relevant information to address user queries. The data supports two subtasks, evidence passage identification and response generation, as well as a human evaluation protocol to assess model performance. We report results of two systems based on state-of-the-art models of conversational knowledge identification and open-domain question answering. Both systems significantly underperform humans, suggesting ample room for improvement in future studies.