Robust Inference in the Presence of Data Heterogeneity and Structured Missing Data

现代测序平台每次运行可以对数百亿个碱基进行测序，并产生千万亿字节的DNA序列。 数据，但个别研究规模可能很小。同样，各种各样的健康数据现在都是公开的， 可用于为卫生政策决策提供信息，并且使用来自几个不同的数据可能是有利的。 调查。跨不同数据集聚合和比较异构数据的能力将是 这对于扩大任何单独研究的可用数据至关重要。我们建议系统地研究 这一努力的两个主要障碍：1）聚集不同的医学和生物数据集; 2）处理 批量效果和结构化异构数据。目标1使我们能够充分利用信息， 不同数据集的相关主题，因为不同实验的信息需要结合在一起， 统计上严格、可靠的方法--该过程需要充分利用现有信息， 引入偏差，并且仍然是系统的和可重复的。并不是所有的实验都研究同一组 变量/特征，并且组合这些信息是一项重要的任务。第二个目标是让研究人员 处理个体或样本之间的异质性，这种异质性在生物学和医学中普遍存在， 健康数据。例如，测序仪随着时间的推移而不断发展， 技术不能直接与旧系统上采集的样本进行比较，即使数据是在 同一个实验室。这也适用于在不同环境条件下获得的样品。目前， 研究人员被迫要么忽视这些偏见，可能导致违反统计有效性，或 将他们的分析限制在一批样品中产生的数据。这项工作将扩大有用的数据集 可供研究人员在各种各样的领域，并提供方法来比较和综合 不同的数据集。所提出的工作将导致：（1）发展的算法与理论 组合来自具有少量重叠的数据集的信息的性能保证 特征;（2）在存在内-的情况下，制定严格的假设检验统计程序。 群体异质性这些方法特别有助于治疗前/治疗后研究， 包含批次效应，或使用不同的方法在长时间内收集样本的研究 （3）在分子生物学（遗传学）领域的案例研究中实施这些方法 路径假设生成）和人口调查数据用于卫生政策建模。

Modern sequencing platforms can sequence tens of billions of bases per run and generate peta-bytes of data, but individual study sizes may be small. Similarly, a wide variety of health data are now publicly available to inform health policy decisions, and it may be advantageous to use data from several different surveys. The ability to aggregate and compare heterogeneous data across different datasets would be critical to expanding the usable data available for any individual study. We propose systematically studying two major barriers to this effort: 1) Aggregating different medical and biological datasets; 2) Dealing with batch effects and structured heterogeneous data. Aim 1 allows us to fully utilize information on related topics from diverse datasets, as information across different experiments needs to be combined in a statistically rigorous, reliable way - the process needs to fully exploit the available information, not introduce biases, and still be systematic and reproducible. Not all experiments study the same set of variables/features, and combining this information is a non-trivial task. The second aim allows researchers to handle heterogeneity between individuals or samples, which happens with ubiquity in biological and health data. For instance, sequencing machines are evolving over time and samples obtained wlth new technologies cannot be directly compared to samples taken on older systems, even if data was collected in the same lab. This also applies to samples obtained under different environmental conditions. Currently, researchers are forced to either ignore such biases, potentially leading to violations of statistical validity, or limit their analysis to data generated in one batch of samples. This work will extend the set of useful data available to researchers in a wide variety of domains and provide methods to compare and synthesize disparate datasets. The proposed work will result in: (1) Development of algorithms with theoretical performance guarantees for combining information from datasets with small number of overlapping features; (2) Development of rigorous statistical procedures for hypothesis testing in the presence of within-. group heterogeneity. These methods are particularly helpful for pre-/post- treatment studies, studies containing batch effects, or studies where samples are collected over long time periods using different technologies; (3) Implementation of these methods in case studies to domains in molecular biology (genetic pathway hypothesis generation) and population survey data for health policy modeling.