The Role Robust Statistical Methods in the Credibility Revolution of Psychological Science

心理学中的可信度运动是对未能复制许多以前被认为是稳健的效果的回应（开放科学合作组织，2015）。以前的研究表明，研究人员在决定如何分析他们的数据时所表现出的灵活性，应用哪种转换，以及保留或排除哪些情况（Steegen et al.，2016年），导致了当前的可复制性问题。分析决策可能会受到许多因素的影响，包括偏向于产生更有可能发表的结果，因为统计意义，但随后可能无法复制。这个项目将调查如何限制分析的灵活性在研究过程中-即通过预登记分析计划和应用强大的统计方法，保持公正的条件下，其他常用的模型表现不佳-可以帮助解决一些问题与心理学的可复制性，减少产生假阳性的影响和提高权力，发现效果，否则将保持未被发现。为此，它将：（1）询问心理学研究人员对一般线性模型（GLM）假设的知识和理解，GLM是心理学中最常用的统计模型，以及他们用于抵消心理学研究数据中常见特征的偏差的解决方案;（2）使用自然语言处理（NLP）从已发表的论文中挖掘信息，并检查研究人员描述数据的语言，统计模型并得出关于调查中的效果的结论;（3）评估在心理数据的真实世界条件下稳健统计方法的性能（例如，典型的分布特征、异常值的存在和有影响的案例等）; (4)评价已发表的预注册和非预注册研究效应的稳健性。该项目将照亮心理学研究中的当前统计实践，并为心理学研究人员在所有职业阶段提供具体的“最佳实践”建议。这项工作将解决在开放科学，可复制性和研究实践的透明度等领域出现的方法论挑战，从而对心理学作为一门科学学科的改进产生直接影响。

The credibility movement in psychology arose as a response to failures to replicate a numberof effects that were previously considered robust (Open Science Collaboration, 2015).Previous research indicated that the flexibility the researchers exercise when deciding howto analyse their data, which transformation to apply, and which cases to retain or exclude(Steegen et al., 2016), has contributed current replicability issues. Analytic decision makingcan be influenced by a number of factors, including bias towards producing results that aremore likely to get published because of statistical significance, but may subsequently fail toreplicate. This project will investigate how limiting analytic flexibility in the research process- namely by preregistering analytic plans and applying robust statistical methods thatremain unbiased under conditions where other commonly used models underperform - canhelp address some of the problems with replicability in psychology by reducing thepossibilities for generating false-positive effects and improving power to discover effectsthat would otherwise remain undetected. It will do so by: (1) querying psychologyresearchers about their knowledge and understanding of the assumptions of the generallinear model (GLM), the most commonly applied statistical model in psychology, and aboutthe solutions they apply to counteract bias from commonly occurring characteristics of datafrom psychological research; (2) using Natural Language Processing (NLP) to text-mineinformation from published papers and examine the language the researchers use todescribe their data, statistical models and draw conclusions about the effects underinvestigation; (3) assessing the performance of robust statistical methods under real-worldconditions of psychological data (e.g., typical distributional characteristics, presence ofoutliers and influential cases, etc.); (4) evaluating the robustness of published effects forpreregistered and non-preregistered studies. The project will shine a light on currentstatistical practice within psychological research and lead to specific 'best practice'recommendations for psychology researchers at all career stages. This work will address theemerging methodological challenges in the areas of Open Science, replicability, andtransparency of research practice, thus having a direct impact on the improvement ofpsychology as a scientific discipline.