Argument Mining for Understanding Peer Reviews
Argument Mining for Understanding Peer Reviews
复制标题
DOI：
10.18653/v1/n19-1219
复制
发表时间：
2019-03
期刊：
影响因子：
--
通讯作者：
Xinyu Hua;M. Nikolov;Nikhil Badugu;Lu Wang
中科院分区：
文献类型：
--
作者：
Xinyu Hua;M. Nikolov;Nikhil Badugu;Lu Wang
同行评审在科学写作和出版生态系统中发挥着至关重要的作用。为了评估审查过程的效率和功效，一个基本要素是理解和评估审查本身。在这项工作中，我们在论点挖掘框架下研究同行评审的内容和结构，通过自动检测（1）审稿人提出的论证命题，以及（2）它们的类型（例如，评估工作或提出改进建议）。我们首先从主要的机器学习和自然语言处理场所收集了 14.2K 条评论。 400 条评论注释有 10,386 个命题以及相应类型的评估、请求、事实、参考或引用。然后，我们在数据上训练最先进的命题分割和分类模型，以评估其效用并确定这个新领域的新挑战，从而激发论点挖掘的未来方向。进一步的实验表明，命题的使用在数量、类型和主题上因场所而异。
Peer-review plays a critical role in the scientific writing and publication ecosystem. To assess the efficiency and efficacy of the reviewing process, one essential element is to understand and evaluate the reviews themselves. In this work, we study the content and structure of peer reviews under the argument mining framework, through automatically detecting (1) the argumentative propositions put forward by reviewers, and (2) their types (e.g., evaluating the work or making suggestions for improvement). We first collect 14.2K reviews from major machine learning and natural language processing venues. 400 reviews are annotated with 10,386 propositions and corresponding types of Evaluation, Request, Fact, Reference, or Quote. We then train state-of-the-art proposition segmentation and classification models on the data to evaluate their utilities and identify new challenges for this new domain, motivating future directions for argument mining. Further experiments show that proposition usage varies across venues in amount, type, and topic.