Modern Statistical Optimization, Robustification and Inference with Applications to Big Data Analytics

点击翻译按钮获取中文摘要

Big data is transforming our world, revolutionizing operations and analytics everywhere, from financial engineering to biomedical sciences. How to efficiently exact useful information from large and noisy datasets is of significant importance, and it has posed at least three grand challenges. The first challenge towards this goal is often computational: the massiveness and complexity of the data call for efficient algorithms to be used in practice. The second challenge arises from the fact that the data collected in real world, big or small, are often contaminated by heavy-tailed errors or outliers, making conventional statistical methods inadequate. Third, an efficient and accurate inference procedure is in demand for big data analytics and valid statistical decision makings. We attempt to addressing these three challenges in this proposal. Our first goal of this proposal is to explore the direction of statistical optimization for nonconvex problems and to understand the hidden convexity in a wide range of continuous and discrete statistical optimization problems. We aim to provide a unified toolbox of algorithms, theories and applications, and we expect to provide new fundamental understanding and tools for optimization in big data. It, upon completed, will have potential and fundamental impact in the area of statistics, machine learning, signal processing, imaging restoration, dictionary learning and artificial intelligence. Our second goal is to comprehensively study nonasymptotic robustification for meaningful data analytics in the presence of low-quality data. The key observation is the bias-robustness tradeoff principal, which we believe is a universal phenomenon in many applications, such as prediction, classification, clustering and inference problems. This goal, upon comprehensively studied, will impact data analytics in practice. We will follow modern software principals in the emphases of compatibility, extendability and maintainability, and develop open-source packages for statisticians and practitioners. Last, we will move forward to modern statistical inference, where we focus on the replicability issues. The scientific question we aim to address is whether the discoveries from a vast statistical search can be replicated in future and independent studies. We will verify the efficacy of our methods by applying them to large-scale imaging genetic datasets, such as the enhancing neuroimaging genetics through meta-analysis and the cohorts for heart and aging research in genomic epidemiology. This goal, upon completed, will impact practitioners in various scientific disciplines, especially in neuroscience and genetics.