Forging Consensus: A Data-Driven Framework for Studying Scientific Consensus and Debate

科学家之间的健康辩论推动科学进步，因为想法和假设之间的竞争鼓励寻找新的证据，并激励科学家考虑他们不同的，往往是分歧的理论和世界观。因此，理解科学需要理解科学辩论在哪里以及为什么会发生。然而，科学辩论的研究历来受到缺乏公认的数据和方法论的挑战。该项目将使用数据驱动的方法来克服这一挑战，该方法利用越来越多的学术活动数据来识别数百万已发表的科学文件中的辩论，并提供围绕研究主题的辩论水平的定量评分。为该项目开发并公开发布的数据和技术将为科学辩论的新科学提供基础，并通过将其应用于研究COVID-19研究中科学辩论的演变来展示其潜力。对科学辩论的深刻理解，包括它们在哪里发生以及为什么发生，需要为科学的理论模型提供信息，为改善科学文献的可获得性提供新的工具，并在科学政策和治理方面做出更好的决定，这将加快科学发现的步伐。首先，使用各种基于启发式和机器学习的技术，将从主要书目和全文数据库中索引的数百万科学出版物中自动识别出一个范例辩论语料库。其次，我们将使用这个语料库开发和严格验证一套主题级的量化指标的辩论，利用国家的最先进的技术应用于出版物元数据，引用链接和全文信息。这些指标中最成功的将被组合成一个数学模型，并用于推断主题的单一辩论得分。这些指标将首次促进对所有科学领域的辩论发生率和共识演变的实证研究。最后，我们将通过使用这些指标来解决与COVID-19大流行对科学的影响有关的政策相关研究问题，来展示这些指标的潜力;具体来说，我们调查了共识在社会知识使用和假新闻发生率中的作用，以及在大流行期间加速科学的实践是否也加速了共识的形成。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Healthy debate between scientists drives scientific progress, as competition between ideas and hypotheses encourages the search for new evidence and motivates scientists to reckon with their different, often diverging theories and worldview. Understanding science, therefore, requires understanding where and why scientific debates occur. However, the study of scientific debate has historically been challenged by a lack of accepted data and methodological approaches. This project will overcome this challenge using a data-driven approach that leverages increasingly-available data on scholarly activity to identify debates across millions of published scientific documents, and to provide a quantitative scoring of the level of debate surrounding a research topic. The data and techniques developed for this project and publicly released will provide a foundation for a new science of scientific debate, and their potential demonstrated by applying them towards studying the evolution of scientific debate within COVID-19 research. A strong understanding of scientific debates, including where they occur and why, is needed to inform theoretical models of science, new tools for improving the accessibility of the scientific literature, and better decisions in science policy and governance that will accelerate the pace of scientific discovery.This project addresses three primary objectives. First, using a variety of heuristic and machine learning based techniques, a corpus of exemplar debates will be automatically identified from among millions of scientific publications indexed in major bibliographic and full-text databases. Second, we will then use this corpus to develop and rigorously validate a suite of topic-level quantitative indicators of debate that leverage state-of-the-art techniques applied to publication metadata, citation linkages, and full-text information. The most successful of these indicators will be combined into a mathematical model and used to infer a singular debate score for topics. These indicators will for the first time facilitate the empirical study of the incidence of debate and evolution of consensus across all of science. Finally, we will demonstrate the potential of these indicators by using them to address policy-relevant research questions relating to the impacts of the COVID-19 pandemic on science; specifically, we investigate the role of consensus in the societal usage of knowledge and the incidence of fake news, and whether practices that accelerated science during the pandemic also accelerated consensus formation. The research will contribute to several fields, including the science of science, science communication, and public policy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.