Hypothesis Testing in High Dimensions Without Sparsity

点击翻译按钮获取中文摘要

The exponential growth of diverse scientific data represents an unprecedented opportunity to make substantial advances in complex science and engineering, such as the discovery of novel materials or drugs. The collection and analysis of information on massive scales has clear benefits for society: it can help businesses optimize online commerce, medical workers address public health issues, and governments interrupt terrorist activities. This project aims at designing rigorous statistical framework and advanced tools for propagating and managing uncertainty in the modeling and design of complex physical and engineering systems. The ultimate goal of this project is to facilitate significant hypothesis generation and accelerate discovery by correlating data across scientific domains in systems with multi-scale and uncertain parameters in extremely high-dimensional spaces, including for example aerospace, engineering, neuroscience, gene-protein disease networks, materials science, climate science, autonomous systems, and cyber-physical systems or self-organized biological systems. This project will enable the design of systems with verifiable properties; reveal how to value the trustworthiness of results in a wide variety of applications; allow conclusions to be probed for their sensitivity to uncertainties and practical experimentation and validation. Understanding the complex and increasingly data-intensive world around us relies on the construction of robust empirical models, i.e., representations of real, complex systems that enable decision makers to predict behaviors and answer "what-if" questions. Novel statistical research is needed for dealing with the underlying high dimensionality of the space of uncertain parameters, active multi-physics coupling, and uncertainty in the models themselves. Besides, there is no fundamental theory for decision making under uncertainty for these large-scale systems. To address these needs, this project intends to develop the following capabilities: new methods for inverse modeling to scale to high-dimensional multi-scale/multi-physics systems; a quantifiable and generalizable understanding of uncertainties and inadequacies in the physical models themselves and entirely new paradigms for decision making that is extremely robust to model misspecification and assumptions. In particular, new decision theory for high-dimensional regression models will be developed that is highly and provably robust to the model misspecification, including missing, non-sparse and large-scale (exploding) data structures. As such, these methods will be the first attempts at understanding the fundamental limitations of dense high-dimensional models. Moreover, censored analysis models (log-rank, additive hazard, proportional hazard and competing risk) will be studied under the setting where perfect estimation or variable selection is not possible; these studies will be able to bridge the gap between the practice and theory in the high-dimensional setting. Furthermore, a new framework for aggregating heterogeneous and disparate data structures that may be correlated and time-dependent will be developed. The added flexibility in analyzing disparate data going from one stationary source to many moving sources of observations must be carefully balanced with parsimony.