Developing user-centric training in rigorous research: post-selection inference, publication bias, and critical evaluation of statistical claims.

项目摘要/摘要 随着科学实践随着数据量的指数增长而发展， 快速计算统计，以及所谓的“再现性危机”，研究人员 开发新的方法，以严格和负责任的方式收集和分析数据。 这项建议的目的是为神经科学研究人员开发三个培训单元。 这将使学习者了解这些开发的速度，提高 通过加深对统计学在生物医学中作用的理解，促进他们的科学研究 研究。在测试、评估和修订的循环中迭代开发的每个单元都将 专为在线或课堂使用而设计，适合不同的学习方式。这些单位将包括一个 一系列短视频片段和互动练习，引导学员了解 当他们朝着一套明确的学习目标前进时，引导他们发现和自我反省。 我们的单元将教授神经科学家在设计和分析数据时避免常见的陷阱。在……里面 在第一个单元中，我们解决了一组容易犯的错误，在这些错误中，研究人员改变了她的计划 在数据分析过程的中途。听话的实践--事后的假设 结果是已知的-包括测试在查看研究后形成的假设 结果。当一项研究基于前一项研究得出负面结果时，就会发生结果转换 具体的结果衡量标准，但报告的是其他衡量标准。《叉子花园》 路径指的是研究人员在进行统计分析时所拥有的自由度 一起去吧。第二个单元处理出版偏见的问题，这一问题在作者 期刊更喜欢发表积极的结果，而不是负面的结果，并可以领导研究人员 重复工作或从已公布的数据中得出错误的推论。本单位的目标是 让学生意识到问题，教他们在阅读文学作品时如何适应，以及 建议在他们自己的工作中避免发表偏见的策略。第三个单位将进行训练 学生如何算出统计分析时是否严谨可靠。学生 将学习如何询问“数据是否适合我们想要了解的内容？”“就是选择 统计检验是否合理？““这些推论是否得到了证据的支持？” 通过开发这套单元，将其包括在更广泛的神经科学课程中，我们可以 培养新一代生物医学科学家，他们装备精良，能够与 由于新的研究工具和技术，数据集正在变得可用。这些 科学家将能够更准确地工作，更有效地做出新发现，以及 以前所未有的速度发展我们在健康和生命科学方面的知识。

Project Summary / Abstract As scientific practice evolves in response to exponential increases in data volume, availability of rapid computational statistics, and the so-called “reproducibility crisis”, researchers are developing new methods for collecting and analyze data in rigorous and responsible fashion. The aim of this proposal is to develop three training units for researchers in the neurosciences that will bring learners up to speed on these developments, improving the rigor and quality of their scientific research by deepening their understanding of the role of statistics in biomedical research. Each unit, developed iteratively in a cycle of testing, evaluation, and revision will be designed for online or classroom use suitable for diverse learning styles. Units will comprise a series of short video segments and interactive exercises that lead learners in a process of guided discovery and self-reflection as they move toward a set of well-specified learning goals. Our units will teach neuroscientists to avoid common pitfalls in designing and analyzing data. In the first unit, we address a set of easy-to-make mistakes wherein a researcher alters her plans midway through the process of data analysis. The practice of HARKing—hypothesizing after the results are known—involves testing hypotheses that are formulated after viewing research outcomes. Outcome switching occurs when a study yields negative results based on the pre- specified outcome measures, but other measures are reported instead. The Garden of Forking Paths refers to the latitude that researchers have in shaping a statistical analysis as they go along. The second unit addresses the problem of publication bias, which arises when authors and journals prefer to publish positive results in favor of negative one, and can lead researchers to reduplicate efforts or draw mistaken inferences from published data. The aim of this unit is to make students aware of problem, teach them how to adjust when reading the literature, and suggest strategies for avoiding publication bias in their own work. The third unit will train students how to figure out whether when a statistical analysis rigorous and reliable. Students will learn how to ask “Are the data appropriate what we want to learn?” “Is the choice of statistical test reasonable?” “Are the inferences supported by the evidence?” By developing this set of units, to be included in a broader neuroscience curriculum, we can train a new generation of biomedical scientists who are well-equipped to work with the vast datasets that are becoming available thanks to new research tools and technologies. These scientists will be able to work more accurately, make new discoveries more efficiently, and advance our knowledge in the health and life sciences at a faster rate than ever before.