SCISIPBIO: A data-science approach to evaluating the likelihood of fraud and error in published studies

科学文献扮演着几个重要的角色。在科学领域，它可以为未来的研究提供信息，为新发现铺平道路，并指导个别科学家的未来计划以及他们如何度过自己的时间和职业。在科学之外，科学文献也发挥着几种作用，比如为政策提供信息或指导个人的司法裁决。出于所有这些原因，保持科学文献的完整性对于科学家、广大公众，以及最终公众对个别科学领域的感知是极其重要的。然而，对于科学家和科学期刊的编辑来说，识别不可信的科学文献仍然很困难。这个项目寻求在可疑的科学手稿公布之前识别它们，使用数据-科学方法，在这种方法中，我们捕捉到科学手稿及其内容的许多明显特征，以及关于作者的信息。该项目的成果将包括一个程序性和基于网络的界面，使决策者和科学期刊等第三方能够扫描手稿，以发现科学欺诈和错误的迹象。该项目将专注于生物医学科学。该界面下的系统包括81个不同的数据库，这些数据库已被汇总、注解(例如，使用所包括的基因的化学和生物特性)，并通过出版元数据(例如，参考文献、作者、资金)进行链接。这些数据将与一个欺诈和错误出版物数据库(使用RetractionWatch和一个人工管理的数据库)相匹配。欺诈性和非欺诈性出版物的特征将以这些数据库为条件，其他特征则以基因和作者的网络特性为基础。该项目将采用不同的机器学习方法，如梯度增强和自动学习器，其性能将在样本外进行评估。第四，为了提高可解释性，更好地理解科学欺诈和错误，并可能提高模型的稳健性，该项目将对模型进行正规化和简化，以将其预测能力减少到一小部分信息。最后，该项目将创建一个基于REST的接口，允许从定制手稿导入。拟议的工作是独一无二的，使手稿具有高度不同的性质，包括内容和世界领先的培训数据。这将为政策制定者和科学编辑提供一个数据驱动的工具，在可疑手稿进入已出版的科学记录之前识别它们。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Scientific literature servers several important roles. Within the sciences it can inform future research, can pave the way toward new discoveries, and guide the future plans of individual scientists and how they spend their own time and careers. Outside of the sciences, scientific literature too serves several roles, such as informing policies or guiding individual judicial decisions. For all of these reasons, maintaining the integrity of the scientific literature is of uttermost importance for scientists, the broad public, and ultimately the public’s perception of individual scientific fields. Yet, identifying non-trustworthy scientific literature even remains difficult for scientists and editors of scientific journals. This project seeks to identify suspicious scientific manuscripts before they are publicized, using a data-scientific approach in which we capture many distinct traits of scientific manuscripts and their content, as well as information about the authors. The outcome of the project will include a programmatic and web-based interface that allowed third parties such as policy makers and scientific journals to scan manuscripts for signs of scientific fraud and error. The project will focus on the biomedical sciences. The system beneath this interface includes 81 distinct databases that have been aggregated, annotated (e.g., with the chemical and biological properties of included genes), and linked through publication metadata (e.g., references, authorship, funding). These data will be matched with a database on fraudulent and erroneous publications (using retractionwatch and a manually curated database). Features of fraudulent and non-fraudulent publications will be conditioned on these databases, with additional features based on network-properties of genes and authors. The project will employ distinct machine learning approaches, such as Gradient Boosting and auto-learners, whose performance will be evaluated out-of-sample. Forth, to improve interpretability, and better understand scientific fraud and error, and possibly improve the robustness of models, the project will regularize and simplify the models to reduce their predictive capabilities to a small set of the information. Lastly, the project will create a REST-based interface that will allow the import from custom manuscripts. The proposed work is unique for conditioning manuscripts on highly distinct properties of manuscripts including content and world-leading training data. This will provide a data-driven tool for policy makers and scientific editors to identify suspicious manuscripts before they enter the published scientific record.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.