SCISIPBIO: Can consultation create a fairer scientific peer-review process?

越来越多的研究表明，偏见可能在科学评审过程中发挥有害作用，特别是代表性不足的群体受到不公平对待。这种偏见破坏了科学的核心手段，即通过同行评审来识别高质量的科学。它还威胁到科学的核心价值：重要的是科学，而不是相关人员的特征。本项目研究同行评议的基本过程，以确定如何减少偏见和如何改进同行评议。我们的研究结果可能会为各个学科的期刊、会议、著名的颁奖机构和资助机构提供信息。在提交稿件的每一个决策时刻，偏见都可能进入同行评审：在最初筛选时，在审稿人提出建议时，甚至在编辑做出最终决定时。通常，这些偏见的例子不公平地偏袒多数群体的作者，因为他们的地位或与审稿人的自我相似性。我们建议通过调查有影响力的开放获取生物医学和生命科学期刊《eLife》的同行评议的各个方面，来确定某些编辑和评议实践是否可以减少这种偏见。在那里，我们可以前所未有地访问所有文本的内容和围绕约35K手稿提交的讨论。我们建议制定各种反映社会偏见、手稿质量、评论和咨询质量的措施，然后在预测建模中使用这些措施来确定减少偏见的科学评论的质量。生命科学的进步依赖于一个健全的审查过程。我们提出的三个项目使用最先进的自然语言处理技术，概念化和测量提交的手稿、评论和同行咨询的实质性质量。一项中心任务是确定审查和协商的实质性质量——特别是对话质量——能否通过减少社会偏见对决策结果可能产生的影响，为更公平的科学进程创造条件。我们对案头决定的研究将帮助我们了解初稿是否以及如何充满问题，以及编辑之外的扩大咨询是否会改善或恶化这些问题。我们对审稿人推荐的研究将帮助我们了解偏见可能渗透到审稿过程中的方式，以及审稿人之间的讨论是否会减少或放大这种偏见。最后，我们对评论和咨询质量的研究将确定科学需要的评论和讨论的种类，以防止偏见的侵入。通过研究科学评论的这些特征，我们将了解如何保护科学客观性免受隐性社会偏见的侵扰，以及可能有助于它的最佳实践。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

A growing body of research shows that bias can play a deleterious role in the scientific review process and that underrepresented groups in particular receive unfair treatment. Such bias undermines science’s central means of advancing through the identification of high-quality science by peer review. It also threatens a core scientific value: that it is the science that matters, not the traits of the people involved. This project studies the fundamental process of peer review to identify how bias may be diminished and how peer review can be improved. Our findings will potentially inform journals across academic disciplines, as well as conferences, prestigious award-granting institutions, and grant-giving bodies. Bias can enter peer review at every decision juncture of a submitted manuscript: at its initial screening, when reviewers make their recommendations, and even when the final decision is made by the editor. Often, these instances of bias unfairly favor authors from majority groups by virtue of their status or self-similarity with reviewers. We propose to determine whether certain editorial and review practices could diminish such bias by investigating all aspects of peer review at an influential open access scientific journal for the biomedical and life sciences, eLife. There, we have unprecedented access to the content of all texts and discussions surrounding ~35K manuscript submissions. We propose developing a variety of measures reflective of social bias, manuscript quality, and the quality of reviews and consultations, and then using those in predictive modeling to identify qualities of scientific review that diminish bias.The advance of science in the life sciences rests on a robust review process. The three projects we propose conceptualize and measure the substantive qualities of submitted manuscripts, reviews, and peer consultations using state-of-the-art techniques from natural language processing. A central task is to identify whether the substantive qualities of reviews and consultations — particularly dialogue qualities — can create conditions for a fairer scientific process by reducing the influence that social bias may have on decision outcomes. Our study of desk decisions will help us understand whether and how the first view of manuscripts can be fraught with problems and whether expanded consultation beyond the editor improves or worsens those problems. Our study of reviewer recommendations will help us understand the ways bias might creep into the review process and whether discussion among reviewers diminishes or amplifies such bias. Last, our study of the qualities of reviews and consultations will identify the sorts of reviews and discussions that science needs to prevent the intrusion of bias. By studying these features of scientific review, we will learn how scientific objectivity can be protected from the intrusion of implicit social bias as well as the best practices that may assist it.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.