Research-Reporting Fidelity within Dietary Weight-Loss Supplement Scientific Literature

 描述（由申请人提供）：研究报告保真度是指在科学论文、演讲和新闻稿中如何分析和呈现数据的决策所涉及的完整性，以便我们的数据和用于生成数据的方法被描述得没有技巧，没有夸张，并且有足够的细节让其他科学家判断和复制我们的方法。健康研究中的一个当前关注的问题是在许多调查领域中研究报告的忠实性受到损害。然而，关于膳食减肥补充剂的文献可能包含一些最普遍和最严重的违规行为。作为一个大多数人都在与肥胖作斗争的国家，饮食减肥补充剂是一种用于减肥的流行资源。基于证据的科学与关于减肥补充剂的结论之间的任何差距都是生物医学研究和公共卫生关注的问题，因为对这些补充剂进行的研究数量和服用此类补充剂的人数。该提案的目的是关注两个特别的问题：（1）通常被称为“p黑客”的现象，定义为使用各种测试重新分析数据，直到获得低于标称α水平的p值，从而增加获得假阳性结果的机会，以及（2）荟萃分析研究中计算效应量的常见错误。该提案旨在评估这些问题的严重程度，从而为评估未来缓解这些问题的进展奠定基线，并测试缓解其中一个问题的教育方案。第一个目的是确定减肥补充文献中的p-hacking的流行程度，通过结合复杂的统计方法，如多重插补和双重抽样，以及通过使用众包，以前所未有的幅度，速度和效率。第二个目的是通过系统地回顾对减肥补充剂进行的荟萃分析，确定在对膳食减肥补充剂进行的同行评审荟萃分析中发生的效应量估计错误的频率。第三个目的是开发和测试三个简短的培训模块的早期职业调查的有效性，旨在提高使用减肥补充荟萃分析的效果大小估计数据的再现性，通过进行一个简单的随机对照试验。该提案旨在促进的方法学进步将提高未来研究人员的技术能力，这些研究人员希望使用多重插补，双重抽样和众包来调查需要从现有科学文献中提取大量详细信息的问题。此外，从该项目中获得的结果将有助于提高膳食减肥补充剂研究的准确性和可重复性，使新手，更广泛的科学界，监管机构，医疗决策者更容易获得准确的科学信息，并有助于改善健康和生活质量。

 DESCRIPTION (provided by applicant): Research reporting fidelity refers to the integrity involved with making decisions about how to analyze and present data in scientific papers, talks, and press releases, such that our data and the methods used to generate them are described without artifice, without exaggeration, and with sufficient detail for other scientists t judge and replicate our methods. A current concern within health research is compromised research- reporting fidelity in many domains of inquiry. However, the literature on dietary weight-loss supplements may contain some of the most pervasive and egregious breaches. As a nation where the majority of individuals struggle with obesity, dietary weight-loss supplements are a popular resource used to lose weight. Any gap between evidence-based science and conclusions drawn about weight-loss supplements is of concern to biomedical research and to public health, given the number of studies conducted on these supplements and the number of persons taking such supplements. The objective of this proposal is to focus on two particular concerns: (1) a phenomenon commonly referred to as "p-hacking" defined as reanalyzing data using various tests until a p-value below the nominal alpha level is obtained, thus inflating the chances of obtaining false positive results, and (2) common errors in the calculation of effect sizes in meta-analytic research. This proposal aims to assess the magnitude of these problems, thereby laying a baseline for the evaluation of future progress in mitigating these problems, and to test an educational program to mitigate one of these problems. The first aim is to determine the prevalence of p-hacking within weight-loss supplement literature with unprecedented magnitude, speed, and efficiency by combining sophisticated statistical methods, such as multiple imputation and double sampling, as well as by employing the use of crowdsourcing. The second aim is to determine the frequency of effect size estimation errors that occur within peer-reviewed meta-analyses conducted on dietary weight-loss supplements, by systematically reviewing meta-analyses conducted on weight-loss supplements. The third aim is to develop and test the efficacy of three brief training modules for early career investigators designed to enhance effect-size estimation data reproducibility using weight-loss supplement meta-analyses, by conducting a simple randomized controlled trial. The methodological advances this proposal aims to foster will enhance the technical capabilities of future researchers who wish to employ the use of multiple imputation, double sampling, and crowdsourcing to investigate questions requiring the extraction of large amounts of detailed information from existing scientific literature. Additionally, the results obtained from this project will help improve the accuracy and reproducibility of dietary weight-loss supplement research, making accurate scientific information more accessible to the novice, the broader scientific community, regulatory agencies, medical decision makers, and for public health to help improve the health and life quality of persons taking, or considering taking, dietary supplements for weight loss.