III:Small: Replication Studies for High Dimensional Data: Insights into Confounding and Heterogeneity

为了让科学界就一项科学发现达成共识，必须在不同小组的多项研究中重复这一发现。不幸的是，并不是所有的科学研究都能成功复制。当一项科学发现被报道时，它的可信度是用p值来量化的。原则上，p值应该量化一项研究的重复频率。在过去的十年里，研究人员已经证明，科学研究的复制速度比报道的p值预测的要低得多。这导致了对复制失败原因的激烈讨论，并制定了研究设计指南，以提高复制速度。在这个项目中，研究小组将展示，当研究收集大量数据时，有可能使用这些数据来确定研究之间的差异，并对为什么研究可以或不可以重复进行获得一些洞察。这些信息可以用来改进个别研究，并提高结果的重复率。由于复制是科学发现的基本工具，开发分析复制研究的新方法将对科学的许多领域产生影响。该团队长期以来一直有兴趣让本科生参与他们的研究，并努力扩大参与者的多样性。在这个项目中，考虑了高维研究的可复制性。在高维研究中，不仅报告了一个p值，而且通常在每个研究中报告了数千甚至数百万个p值。基因组研究是高维研究的一个鼓舞人心的例子，因为基因组数据本身就是高维的，因此在基因组研究中，为每个基因组特征计算p值，例如基因表达水平或遗传变异。通常，在基因组研究中，在所有的p值中，只有一小部分被认为是有意义的(考虑到多次测试)。当执行重复研究时，感兴趣的特征是在原始研究中显著的特征。这个项目背后的关键思想是，有关于所有报告功能的信息，即使是那些不重要的功能。通过对它们的分析，可以获得关于研究的见解，这些见解既可以提高复制速度，也可以提高对每项研究的分析。该框架可用于解决以下问题：(1)减少每次重复中混杂因素的影响--提高能力和减少假阳性；(2)说明报告结果中的确定偏差；(3)解释每次重复或研究之间的差异，以深入了解差异的根本原因。该方法将使用5个基因组数据集进行评估。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

In order for the scientific community to reach consensus on a scientific finding, the finding must be replicated in multiple studies by different groups. Unfortunately, not all scientific studies successfully replicate. When a scientific finding is reported, its confidence is quantified by a p-value. In principal, p- values should quantify how often a study should replicate. Over the past decade, researchers have shown that scientific studies replicate at a much lower rate than the reported p-values predict. This has led to a vigorous discussion on the causes of replication failures as well as developing guidelines for study design to improve the replication rate. In this project, the research team will show that when studies are collecting large amounts of data, it is possible to use this data to identify differences between the studies and gain some insight into why studies do or do not replicate. This information can be used to improve the individual studies and increase the replication rate of the resulting findings. As replication is a fundamental tool in scientific discovery, developing new approaches to analyzing replication studies will have an impact in many areas of science. The team has a long standing interest in involving undergraduate students in their research as well as working to broaden the diversity of participants.In this project, the replicability of high dimensional studies is considered. In a high dimensional study, not only one p-value is reported, but typically thousands or even millions of p- values are reported in each study. Genomic studies are a motivating example of high dimensional studies as genomic data is inherently high dimensional and thus in genomic studies, a p-value is computed for each genomic features such as a gene expression level or genetic variant. Typically, in a genomic study, out of all of the p-values, only a small subset of them are considered significant (taking into account for multiple testing). When a replication study is performed, the features of interest are the features which were significant in the original study. The key idea behind this project is that there is information on all reported features, even those that are not significant. By analyzing them, insights can be obtained about the studies and these insights can both improve the replication rate as well as the analysis of each of the studies. The framework can be leveraged to address the following problems: (1) Reduce the effect of confounders in each replicate -- improving power and reducing false positives; (2) Accounting for ascertainment biases in the reported results; and (3) Interpreting the differences between each replicate or study to gain insights into the underlying causes of the difference. The approach will be evaluated using 5 genomic datasets.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.