SCISIPBIO: Can consultation create a fairer scientific peer-review process?

越来越多的研究表明，偏见可能在科学审查过程中发挥有害作用，特别是代表性不足的群体受到不公平待遇。这种偏见破坏了通过同行评审确定高质量科学来推进科学的核心手段。它还威胁到一个核心的科学价值观：重要的是科学，而不是相关人员的特征。本研究旨在探讨同侪互评的基本程序，以找出如何减少偏见及如何改善同侪互评。我们的研究结果可能会为跨学科的期刊，以及会议，着名的颁奖机构和赠款机构提供信息。偏见可以在提交稿件的每个决策时刻进入同行评审：在最初的筛选中，当审稿人提出建议时，甚至当编辑做出最终决定时。通常，这些偏见的例子不公平地有利于大多数群体的作者，因为他们的地位或与审稿人的自相似性。我们建议，以确定是否某些编辑和审查的做法可以减少这种偏见，通过调查的各个方面的同行评议在一个有影响力的开放获取的生物医学和生命科学，eLife科学期刊。在那里，我们可以前所未有地访问所有文本的内容和围绕约35 K手稿提交的讨论。我们建议开发一系列反映社会偏见、稿件质量、审稿质量和咨询质量的指标，然后将其用于预测模型中，以确定减少偏见的科学审稿质量。生命科学领域的科学进步依赖于稳健的审稿过程。我们提出的三个项目概念化和衡量提交的手稿，评论和同行协商的实质性质量，使用自然语言处理的最先进技术。一项中心任务是确定审查和协商的实质性质量-特别是对话质量-是否能够通过减少社会偏见可能对决策结果的影响，为更公平的科学进程创造条件。我们对案头决定的研究将有助于我们了解手稿的第一眼是否以及如何充满问题，以及编辑之外的扩展咨询是否会改善或消除这些问题。我们对审稿人推荐的研究将帮助我们了解偏见可能会如何渗透到审稿过程中，以及审稿人之间的讨论是减少还是放大了这种偏见。最后，我们对审查和咨询质量的研究将确定科学需要哪些类型的审查和讨论，以防止偏见的入侵。通过研究这些特点的科学审查，我们将学习如何科学的客观性，可以保护从入侵的隐含的社会偏见，以及最佳实践，可能有助于它。这个奖项反映了NSF的法定使命，并已被认为是值得通过评估使用基金会的智力价值和更广泛的影响审查标准的支持。

A growing body of research shows that bias can play a deleterious role in the scientific review process and that underrepresented groups in particular receive unfair treatment. Such bias undermines science’s central means of advancing through the identification of high-quality science by peer review. It also threatens a core scientific value: that it is the science that matters, not the traits of the people involved. This project studies the fundamental process of peer review to identify how bias may be diminished and how peer review can be improved. Our findings will potentially inform journals across academic disciplines, as well as conferences, prestigious award-granting institutions, and grant-giving bodies. Bias can enter peer review at every decision juncture of a submitted manuscript: at its initial screening, when reviewers make their recommendations, and even when the final decision is made by the editor. Often, these instances of bias unfairly favor authors from majority groups by virtue of their status or self-similarity with reviewers. We propose to determine whether certain editorial and review practices could diminish such bias by investigating all aspects of peer review at an influential open access scientific journal for the biomedical and life sciences, eLife. There, we have unprecedented access to the content of all texts and discussions surrounding ~35K manuscript submissions. We propose developing a variety of measures reflective of social bias, manuscript quality, and the quality of reviews and consultations, and then using those in predictive modeling to identify qualities of scientific review that diminish bias.The advance of science in the life sciences rests on a robust review process. The three projects we propose conceptualize and measure the substantive qualities of submitted manuscripts, reviews, and peer consultations using state-of-the-art techniques from natural language processing. A central task is to identify whether the substantive qualities of reviews and consultations — particularly dialogue qualities — can create conditions for a fairer scientific process by reducing the influence that social bias may have on decision outcomes. Our study of desk decisions will help us understand whether and how the first view of manuscripts can be fraught with problems and whether expanded consultation beyond the editor improves or worsens those problems. Our study of reviewer recommendations will help us understand the ways bias might creep into the review process and whether discussion among reviewers diminishes or amplifies such bias. Last, our study of the qualities of reviews and consultations will identify the sorts of reviews and discussions that science needs to prevent the intrusion of bias. By studying these features of scientific review, we will learn how scientific objectivity can be protected from the intrusion of implicit social bias as well as the best practices that may assist it.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.