III: Small: Causal and Statistical Inference in the Presence of Confounding Factors

技术：如果混杂因素与观察数据相关，则存在未测量的混杂因素可能导致不正确的统计和因果推断。这种现象至少在两个重要的应用中得到了很好的证明。一个应用是从相关个体的群体中识别与疾病有关的遗传变异。第二个应用是在比较疾病和健康样本时识别在疾病中活跃的基因。在这个建议中，我们提出了一种新的方法来纠正未观察到的混杂因素，利用混杂因素如何影响高维数据的见解。这些见解激发了一个正式的定义，为特定类型的混杂因素，我们称之为“低秩混杂因素。“将这一定义形式化，使我们能够激励纠正这些类型混杂因素影响的方法，即使在没有观察到混杂因素的情况下。我们的建议将发展一个理论，如何混杂影响数据，在什么条件下未观察到的混杂可以被纠正。所提出的理论是有关最近的事态发展，了解稀疏性已在电气工程，计算机科学和统计学研究。我们所提出的方法的结果将导致改进的方法的应用程序中，这样的混淆present.Non-technical：从高维数据的知识推理是一个基本的问题，影响几乎所有领域的科学，包括物理学，天文学，化学，计算机科学，社会科学和生物学的许多领域。其中许多问题是由最近可获得的大量数据来源以及测量或数据收集技术的进步造成的。一个主要的挑战是存在未知的（和未测量的）混杂因素。混杂因素是在数据中经常观察不到的变量，但与数据的各种特征相关。不幸的是，混杂因素可能导致不正确的推断。这种现象在至少两个重要的应用中得到了很好的证明：一个应用是从相关个体的群体中识别与疾病有关的遗传变异，第二个应用是在比较疾病和健康样本时识别疾病中的活性基因。如果在数据中观察到混杂因素，则有传统的方法来执行推断。然而，处理未观察到的混杂因素更为困难。该项目将开发和研究一种新的方法来纠正未观察到的混杂因素，利用混杂因素如何影响高维数据的见解。该项目具有广泛的影响，由于其在广泛的科学问题的效用，通过跨学科的研究机会提供给本科生和研究生，并通过软件和数据的分布。

Technical:The presence of unmeasured confounding factors can result in incorrect statistical and causal inferences if the confounding factors are correlated with the observed data. This phenomenon has been well documented in at least two important applications. One application is identifying genetic variation involved in disease from populations of related individuals. A second application is identifying genes active in a disease when comparing disease and health samples. In this proposal we propose a new approach to correct for unobserved confounders in taking advantage of insights into how confounders affect high dimensional data. These insights motivate a formal definition for a specific type of confounder which we term a 'low-rank confounder.' Formalizing this definition allows us to motivate methods for correcting for the effects of these types confounders even when the confounders are not observed. Our proposal will develop a theory of how confounders affect data and under what conditions unobserved confounders can be corrected. The proposed theory is related to recent developments in understanding sparsity which has been well studied in electrical engineering, computer science and statistics. The result of our proposed methods will lead to improved methods for applications where such confounders are present.Non-technical:Inference of knowledge from high dimensional data is a fundamental problem affecting virtually all areas of science including physics, astronomy, chemistry, computer science, social science and many areas of biology. Many of these problems are driven by recently available large sources of data and advances in measurement or data collection technologies. A major challenge is the presence of unknown (and unmeasured) confounding factors. Confounding factors are variables that are often not observed in the data, but are correlated with various features of the data. Unfortunately, confounding factors can cause incorrect inferences. This phenomenon has been well documented in at least two important applications: one application is identifying genetic variation involved in disease from populations of related individuals, and a second application is identifying genes active in a disease when comparing disease and health samples. There are traditional approaches to perform inference if the confounders are observed in the data. However, dealing with unobserved confounders is more difficult. This project will develop and study a new approach to correct for unobserved confounders, taking advantage of insights into how confounders affect high dimensional data. The project has broad impact due to its utility in a wide range of scientific questions, through the interdisciplinary research opportunities provided to undergraduate and graduate students, and through the distribution of software and data.