SCISIPBIO: Can consultation create a fairer scientific peer-review process?

越来越多的研究表明，偏见可能在科学审查过程中发挥有害作用，特别是代表不足的群体受到不公平待遇。这种偏见破坏了科学通过同行评议确定高质量科学而推进的核心手段。它还威胁到一个核心的科学价值：重要的是科学，而不是相关人员的特征。本项目研究同行评议的基本过程，以确定如何减少偏见和如何改进同行评议。我们的发现可能会为学术领域的期刊以及会议、著名的颁奖机构和拨款机构提供信息。偏见可以在提交的稿件的每一个决定时刻进入同行评议：在最初的筛选，当评审员提出他们的建议，甚至当最后的决定由编辑做出。通常，这些偏见的例子不公平地偏袒来自多数群体的作者，因为他们的地位或与评审者的自我相似。我们建议通过调查生物医学和生命科学的有影响力的开放获取科学期刊eLife的同行审查的所有方面，来确定某些编辑和审查做法是否可以减少这种偏见。在那里，我们可以史无前例地访问所有文本的内容和围绕~35K手稿提交的讨论。我们建议开发一系列反映社会偏见、稿件质量以及审查和咨询质量的指标，然后使用预测模型中的这些指标来确定减少偏见的科学审查质量。生命科学的进步依赖于强大的审查过程。我们提出的三个项目利用自然语言处理的最新技术，对提交的手稿、审查和同行咨询的实质性质量进行概念化和衡量。一项中心任务是确定审查和协商的实质性性质--特别是对话性质--是否能够通过减少社会偏见对决策结果的影响，为更公平的科学进程创造条件。我们对案头决定的研究将有助于我们理解第一次查看手稿是否以及如何充满问题，以及超出编辑范围的广泛咨询是改善还是恶化这些问题。我们对评审员建议的研究将帮助我们了解偏见可能潜入审查过程的方式，以及评审员之间的讨论是否会减少或放大这种偏见。最后，我们对评论和咨询质量的研究将确定科学需要哪些类型的评论和讨论来防止偏见的侵入。通过研究科学审查的这些特点，我们将了解如何保护科学客观性免受隐性社会偏见的入侵，以及可能有助于科学客观性的最佳实践。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

A growing body of research shows that bias can play a deleterious role in the scientific review process and that underrepresented groups in particular receive unfair treatment. Such bias undermines science’s central means of advancing through the identification of high-quality science by peer review. It also threatens a core scientific value: that it is the science that matters, not the traits of the people involved. This project studies the fundamental process of peer review to identify how bias may be diminished and how peer review can be improved. Our findings will potentially inform journals across academic disciplines, as well as conferences, prestigious award-granting institutions, and grant-giving bodies. Bias can enter peer review at every decision juncture of a submitted manuscript: at its initial screening, when reviewers make their recommendations, and even when the final decision is made by the editor. Often, these instances of bias unfairly favor authors from majority groups by virtue of their status or self-similarity with reviewers. We propose to determine whether certain editorial and review practices could diminish such bias by investigating all aspects of peer review at an influential open access scientific journal for the biomedical and life sciences, eLife. There, we have unprecedented access to the content of all texts and discussions surrounding ~35K manuscript submissions. We propose developing a variety of measures reflective of social bias, manuscript quality, and the quality of reviews and consultations, and then using those in predictive modeling to identify qualities of scientific review that diminish bias.The advance of science in the life sciences rests on a robust review process. The three projects we propose conceptualize and measure the substantive qualities of submitted manuscripts, reviews, and peer consultations using state-of-the-art techniques from natural language processing. A central task is to identify whether the substantive qualities of reviews and consultations — particularly dialogue qualities — can create conditions for a fairer scientific process by reducing the influence that social bias may have on decision outcomes. Our study of desk decisions will help us understand whether and how the first view of manuscripts can be fraught with problems and whether expanded consultation beyond the editor improves or worsens those problems. Our study of reviewer recommendations will help us understand the ways bias might creep into the review process and whether discussion among reviewers diminishes or amplifies such bias. Last, our study of the qualities of reviews and consultations will identify the sorts of reviews and discussions that science needs to prevent the intrusion of bias. By studying these features of scientific review, we will learn how scientific objectivity can be protected from the intrusion of implicit social bias as well as the best practices that may assist it.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.