Forging Consensus: A Data-Driven Framework for Studying Scientific Consensus and Debate

科学家之间的健康辩论推动了科学进步，因为想法和假设之间的竞争鼓励寻找新的证据，并激励科学家考虑他们不同的、往往存在分歧的理论和世界观。因此，理解科学需要理解科学辩论发生的地点和原因。然而，科学辩论的研究历来因缺乏公认的数据和方法论方法而受到挑战。该项目将使用一种数据驱动的方法来克服这一挑战，该方法利用日益可用的学术活动数据来识别数百万已发表的科学文献中的辩论，并提供围绕研究主题的辩论水平的量化评分。为该项目开发并公开发布的数据和技术将为一门新的科学辩论科学提供基础，并通过将它们应用于研究新冠肺炎研究中科学辩论的演变来展示其潜力。需要对科学辩论有深刻的了解，包括它们在哪里发生以及为什么发生，以便为科学的理论模型提供信息，为改善科学文献的可及性提供新的工具，并在科学政策和管理方面做出更好的决策，以加快科学发现的步伐。这个项目涉及三个主要目标。首先，使用各种启发式和基于机器学习的技术，将从主要书目和全文数据库索引的数百万种科学出版物中自动确定范例辩论语料库。其次，我们将使用这个语料库来开发和严格验证一套主题级别的辩论量化指标，这些指标利用应用于出版元数据、引文链接和全文信息的最先进技术。这些指标中最成功的将被组合成一个数学模型，并用来推断主题的单一辩论分数。这些指标将首次促进对整个科学界辩论的发生率和共识演变的实证研究。最后，我们将利用这些指标来解决与新冠肺炎大流行对科学的影响有关的与政策相关的研究问题，从而展示这些指标的潜力；具体地说，我们调查共识在社会使用知识和假新闻发生率方面的作用，以及在大流行期间加速科学的做法是否也加速了共识的形成。这项研究将在几个领域做出贡献，包括科学、科学传播和公共政策。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Healthy debate between scientists drives scientific progress, as competition between ideas and hypotheses encourages the search for new evidence and motivates scientists to reckon with their different, often diverging theories and worldview. Understanding science, therefore, requires understanding where and why scientific debates occur. However, the study of scientific debate has historically been challenged by a lack of accepted data and methodological approaches. This project will overcome this challenge using a data-driven approach that leverages increasingly-available data on scholarly activity to identify debates across millions of published scientific documents, and to provide a quantitative scoring of the level of debate surrounding a research topic. The data and techniques developed for this project and publicly released will provide a foundation for a new science of scientific debate, and their potential demonstrated by applying them towards studying the evolution of scientific debate within COVID-19 research. A strong understanding of scientific debates, including where they occur and why, is needed to inform theoretical models of science, new tools for improving the accessibility of the scientific literature, and better decisions in science policy and governance that will accelerate the pace of scientific discovery.This project addresses three primary objectives. First, using a variety of heuristic and machine learning based techniques, a corpus of exemplar debates will be automatically identified from among millions of scientific publications indexed in major bibliographic and full-text databases. Second, we will then use this corpus to develop and rigorously validate a suite of topic-level quantitative indicators of debate that leverage state-of-the-art techniques applied to publication metadata, citation linkages, and full-text information. The most successful of these indicators will be combined into a mathematical model and used to infer a singular debate score for topics. These indicators will for the first time facilitate the empirical study of the incidence of debate and evolution of consensus across all of science. Finally, we will demonstrate the potential of these indicators by using them to address policy-relevant research questions relating to the impacts of the COVID-19 pandemic on science; specifically, we investigate the role of consensus in the societal usage of knowledge and the incidence of fake news, and whether practices that accelerated science during the pandemic also accelerated consensus formation. The research will contribute to several fields, including the science of science, science communication, and public policy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.