SemEval-2021 Task 9: Fact Verification and Evidence Finding for Tabular Data in Scientific Documents (SEM-TAB-FACTS)
SemEval-2021 Task 9: Fact Verification and Evidence Finding for Tabular Data in Scientific Documents (SEM-TAB-FACTS)
复制标题
SemEval-2021 任务 9：科学文档中表格数据的事实验证和证据查找 (SEM-TAB-FACTS)
DOI：
10.18653/v1/2021.semeval-1.39
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Sara Rosenthal
中科院分区：
文献类型：
--
作者：
N. Wang;Diwakar Mahajan;Marina Danilevsky;Sara Rosenthal
理解表格是一项重要且相关的任务，涉及理解表格结构以及能够比较和对比单元格内的信息。在本文中，我们通过提出一个新的数据集和任务来解决这一挑战，这些任务在 SemEval 2020 任务 9 的共享任务中实现了这一目标：科学文档中表格数据的事实验证和证据查找 (SEM-TAB-FACTS)。我们的数据集包含 981 个手动生成的表格和一个由 1980 个表格组成的自动生成的数据集，提供超过 180K 条语句和超过 1600 万条证据注释。 SEM-TAB-FACTS 有两个子任务。在子任务 A 中，目标是确定与表相关的陈述是否得到支持、反驳或未知。在子任务 B 中，重点是识别表格中为该陈述提供证据的特定单元格。 69 个团队报名参加该任务，其中 19 个团队成功提交了子任务 A，12 个团队成功提交了子任务 B。我们展示了我们的结果和竞赛的主要发现。
Understanding tables is an important and relevant task that involves understanding table structure as well as being able to compare and contrast information within cells. In this paper, we address this challenge by presenting a new dataset and tasks that addresses this goal in a shared task in SemEval 2020 Task 9: Fact Verification and Evidence Finding for Tabular Data in Scientific Documents (SEM-TAB-FACTS). Our dataset contains 981 manually-generated tables and an auto-generated dataset of 1980 tables providing over 180K statement and over 16M evidence annotations. SEM-TAB-FACTS featured two sub-tasks. In sub-task A, the goal was to determine if a statement is supported, refuted or unknown in relation to a table. In sub-task B, the focus was on identifying the specific cells of a table that provide evidence for the statement. 69 teams signed up to participate in the task with 19 successful submissions to subtask A and 12 successful submissions to subtask B. We present our results and main findings from the competition.