Reproducibility and simulation of interactive retrieval experiments (RESIRE)

交互式检索实验的再现性和模拟（RESIRE）

• 批准号: 509543643
• 负责人: Professor Dr. Norbert Fuhr
• 金额: --
• 依托单位: Fachgebiet Informationssysteme
• 依托单位国家: 德国
• 项目类别: Research Grants
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/509543643?language=en
• 关键词: Reproducibility; simulation; interactive; retrieval; experiments

Although reproducibility of experiments is a topic with increasing importance in information retrieval (IR), there has been hardly any work on reproducing interactive IR experiments. The main idea of our proposed project is to describe actual users by quantitative modeling of their behavior and their content-related decisions during retrieval sessions. These model parameters can then form the basis for comparing two interactive IR experiments with regard to reproducibility, and can also be used for simulating interactive IR, e. g. for evaluating new systems or system variants without (or before) carrying out actual user experiments. As part of the proposed project, we will define a new framework model for reproducibility of interactive IR experiments, develop new metrics for reproducibility and session quality, design new simulation methods integrating behavior and content-related decisions, develop an open source framework for reproduction and simulation of interactive IR, and perform extensive experiments using existing IR log data as well as observation data from new user experiments.

实验的再现性是信息检索中越来越重要的一个课题，但交互式信息检索实验的再现性研究还很少。我们提出的项目的主要思想是描述实际用户的行为和他们的内容相关的决策在检索会话的定量建模。然后，这些模型参数可以形成用于比较两个交互式IR实验的再现性的基础，并且还可以用于模拟交互式IR，例如。G.用于评估新系统或系统变体，而无需（或在此之前）执行实际用户实验。作为拟议项目的一部分，我们将定义一个新的框架模型的再现性的交互式IR实验，开发新的指标的再现性和会话质量，设计新的模拟方法集成的行为和内容相关的决定，开发一个开源框架的再现和模拟的交互式IR，并进行广泛的实验，使用现有的IR日志数据以及观察数据从新的用户实验。