Scientists' perception of pilot study quality was influenced by statistical significance and study design.
Scientists' perception of pilot study quality was influenced by statistical significance and study design.
复制标题
科学家对试点研究质量的看法受到统计显着性和研究设计的影响。
DOI：
10.1016/j.jclinepi.2023.05.011
复制
发表时间：
2023
影响因子：
7.2
通讯作者：
Beets,MichaelW
中科院分区：
文献类型：
--
作者：
vonKlinggraeff,Lauren;Burkart,Sarah;Pfledderer,ChristopherD;SabaNishat,MdNasim;Armstrong,Bridget;Weaver,RGlenn;McLain,AlexanderC;Beets,MichaelW
初步研究在制定大规模干预措施中发挥着关键作用，但由于其初步研究状态，在同行评议过程中可能会被要求更高或更低的科学标准。研究设计和背景对5项已发表的肥胖预防初步研究的摘要进行系统修改，使每个摘要产生16个变体。差异由4个因素决定：样本量（n= 20 vs.n= 150）、统计学意义（P< 0.05 vs.P < 0.05）、研究设计（单组vs.随机2组）和初步研究状态（有无先导语言）。通过一项在线调查，行为科学家们随机选择了这5篇摘要中每一篇的变体，并且对其他变体的存在视而不见。受访者对每个摘要的研究质量进行了评分。结果行为科学家（n= 271，女性79.7%，中位年龄34岁）共完成了1355份摘要评分。初步研究状态与感知研究质量无关。统计上显著的效果被评为更具有科学意义、严谨、创新、清晰的书写、值得进一步测试，并且具有更有意义的结果。随机设计被认为更严谨、更创新、更有意义。结论：研究结果表明，审稿人更重视统计上显著的发现和随机对照设计，而可能忽略了其他重要的研究特征。
ObjectivesPreliminary studies play a key role in developing large-scale interventions but may be held to higher or lower scientific standards during the peer review process because of their preliminary study status.Study Design and SettingAbstracts from 5 published obesity prevention preliminary studies were systematically modified to generate 16 variations of each abstract. Variations differed by 4 factors: sample size (n= 20 vs.n= 150), statistical significance (P< 0.05 vs.P> 0.05), study design (single group vs. randomized 2 groups), and preliminary study status (presence/absence of pilot language). Using an online survey, behavioral scientists were provided with a randomly selected variation of each of the 5 abstracts and blinded to the existence of other variations. Respondents rated each abstract on aspects of study quality.ResultsBehavioral scientists (n= 271, 79.7% female, median age 34 years) completed 1,355 abstract ratings. Preliminary study status was not associated with perceived study quality. Statistically significant effects were rated as more scientifically significant, rigorous, innovative, clearly written, warranted further testing, and had more meaningful results. Randomized designs were rated as more rigorous, innovative, and meaningful.ConclusionFindings suggest reviewers place a greater value on statistically significant findings and randomized control design and may overlook other important study characteristics.