Alignment Rationale for Query-Document Relevance
Alignment Rationale for Query-Document Relevance
复制标题
DOI：
10.1145/3477495.3531883
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Youngwoo Kim;Razieh Rahimi;J. Allan
中科院分区：
文献类型：
--
作者：
Youngwoo Kim;Razieh Rahimi;J. Allan
深度神经网络广泛应用于文本对分类任务，如特殊信息检索。这些深度神经网络本身是不可解释的，需要额外的努力来获得其决策背后的基本原理。现有的解释模型还不能诱导查询术语和文档术语之间的对齐——文档基本原理的哪一部分负责查询的哪一部分？在本文中，我们研究了如何使用输入扰动来推断或评估查询和文档跨度之间的对齐，这最好地解释了黑箱排名器的相关性预测。我们使用不同的扰动策略，并相应地提出了一组指标来评估对齐原理对模型的忠实度。我们的实验表明，与基于删除的度量相比，基于替换的扰动的定义度量在选择更高质量的对齐方面更成功。
Deep neural networks are widely used for text pair classification tasks such as as adhoc information retrieval. These deep neural networks are not inherently interpretable and require additional efforts to get rationale behind their decisions. Existing explanation models are not yet capable of inducing alignments between the query terms and the document terms -- which part of the document rationales are responsible for which part of the query? In this paper, we study how the input perturbations can be used to infer or evaluate alignments between the query and document spans, which best explain the black-box ranker's relevance prediction. We use different perturbation strategies and accordingly propose a set of metrics to evaluate the faithfulness of alignment rationales to the model. Our experiments show that the defined metrics based on substitution-based perturbation are more successful in preferring higher-quality alignments, compared to the deletion-based metrics.