TREC-COVID: rationale and structure of an information retrieval shared task for COVID-19
TREC-COVID: rationale and structure of an information retrieval shared task for COVID-19
复制标题
DOI：
10.1093/jamia/ocaa091
复制
发表时间：
2020-09-01
影响因子：
6.4
通讯作者：
Hersh, William R.
中科院分区：
文献类型：
--
作者：
Roberts, Kirk;Alam, Tasmeer;Hersh, William R.
TREC-COVID是一个信息检索(IR)共享任务，在新冠肺炎大流行期间发起，以支持临床医生和临床研究。大流行的IR打破了许多正常的假设，这可以通过检查与大流行情况有关的9个重要的基本IR研究问题来看出。TREC-COVID不同于传统的IR共享任务评估，它特别考虑了预期用户、IR模式考虑因素、主题开发、参与者要求、评估过程、相关性标准、评估指标、迭代过程、预计时间表以及数据用作任务后测试收集的影响。本文描述了在大流行情况下，如何针对开发IR系统的特殊要求处理所有这些问题。最后，还提供了最初的参与数字，这表明IR社区对这一努力有着极大的兴趣。
TREC-COVID is an information retrieval (IR) shared task initiated to support clinicians and clinical research during the COVID-19 pandemic. IR for pandemics breaks many normal assumptions, which can be seen by examining 9 important basic IR research questions related to pandemic situations. TREC-COVID differs from traditional IR shared task evaluations with special considerations for the expected users, IR modality considerations, topic development, participant requirements, assessment process, relevance criteria, evaluation metrics, iteration process, projected timeline, and the implications of data use as a post-task test collection. This article describes how all these were addressed for the particular requirements of developing IR systems under a pandemic situation. Finally, initial participation numbers are also provided, which demonstrate the tremendous interest the IR community has in this effort.