Developing user-centric training in rigorous research: post-selection inference, publication bias, and critical evaluation of statistical claims.

项目总结/摘要 随着科学实践的发展，以应对数据量的指数增长， 快速计算统计，以及所谓的“再现性危机”，研究人员正在 以严谨和负责任的方式开发收集和分析数据的新方法。 该建议的目的是为神经科学研究人员开发三个培训单元 这将使学习者加快这些发展，提高严谨性和质量， 通过加深对统计在生物医学中的作用的理解， research.在测试、评估和修订的循环中迭代开发的每个单元将 专为在线或课堂使用而设计，适合不同的学习风格。单位将包括一个 一系列简短的视频片段和互动练习，引导学习者 引导发现和自我反思，因为他们朝着一套明确的学习目标。 我们的单位将教神经科学家，以避免在设计和分析数据的常见陷阱。在 在第一单元，我们讨论了一组容易犯的错误，其中研究人员改变了她的计划 在数据分析过程的中途。哈金假说的实践 结果是已知的-包括测试的假设，是制定后，观看研究 结果。结果转换发生在研究根据预测试结果得出阴性结果时。 指定的结果指标，但报告的是其他指标。分叉花园 路径是指研究人员在进行统计分析时所拥有的自由度 沿着。第二个单元解决了出版偏见的问题，这是当作者 期刊更喜欢发表积极的结果，而不是消极的结果， 重复努力或从公布的数据中得出错误的推论。本单元的目的是 让学生意识到问题，教他们如何在阅读文献时进行调整， 建议策略，以避免出版偏见，在自己的工作。第三部队将训练 学生在统计分析时如何弄清楚是否严谨可靠。学生 将学习如何问“数据是否适合我们想要学习的内容？”“是选择 统计测试合理吗？”“推论有证据支持吗？” 通过开发这套单元，将其纳入更广泛的神经科学课程，我们可以 培养新一代的生物医学科学家，他们有能力与广大的 由于新的研究工具和技术，数据集正在变得可用。这些 科学家将能够更准确地工作，更有效地进行新的发现， 以比以往任何时候都快的速度推进我们在健康和生命科学方面的知识。

Project Summary / Abstract As scientific practice evolves in response to exponential increases in data volume, availability of rapid computational statistics, and the so-called “reproducibility crisis”, researchers are developing new methods for collecting and analyze data in rigorous and responsible fashion. The aim of this proposal is to develop three training units for researchers in the neurosciences that will bring learners up to speed on these developments, improving the rigor and quality of their scientific research by deepening their understanding of the role of statistics in biomedical research. Each unit, developed iteratively in a cycle of testing, evaluation, and revision will be designed for online or classroom use suitable for diverse learning styles. Units will comprise a series of short video segments and interactive exercises that lead learners in a process of guided discovery and self-reflection as they move toward a set of well-specified learning goals. Our units will teach neuroscientists to avoid common pitfalls in designing and analyzing data. In the first unit, we address a set of easy-to-make mistakes wherein a researcher alters her plans midway through the process of data analysis. The practice of HARKing—hypothesizing after the results are known—involves testing hypotheses that are formulated after viewing research outcomes. Outcome switching occurs when a study yields negative results based on the pre- specified outcome measures, but other measures are reported instead. The Garden of Forking Paths refers to the latitude that researchers have in shaping a statistical analysis as they go along. The second unit addresses the problem of publication bias, which arises when authors and journals prefer to publish positive results in favor of negative one, and can lead researchers to reduplicate efforts or draw mistaken inferences from published data. The aim of this unit is to make students aware of problem, teach them how to adjust when reading the literature, and suggest strategies for avoiding publication bias in their own work. The third unit will train students how to figure out whether when a statistical analysis rigorous and reliable. Students will learn how to ask “Are the data appropriate what we want to learn?” “Is the choice of statistical test reasonable?” “Are the inferences supported by the evidence?” By developing this set of units, to be included in a broader neuroscience curriculum, we can train a new generation of biomedical scientists who are well-equipped to work with the vast datasets that are becoming available thanks to new research tools and technologies. These scientists will be able to work more accurately, make new discoveries more efficiently, and advance our knowledge in the health and life sciences at a faster rate than ever before.