Optimization and Statistical Procedures for Big Data and Applications

在大数据时代，数据集可能是海量的，具有高维。例如，社交媒体数据、高分辨率图像数据和基因组数据。高维和海量样本给计算和统计带来了巨大的挑战。该项目旨在为在重要应用中收集的大数据研究新的优化技术和统计分析工具。该项目将显著增强大数据分析的优化和统计能力。该项目的成果预计将惠及公共卫生、医学研究和金融投资组合管理等广泛领域。该项目由三个子项目组成，以促进现代优化技术和大数据统计程序的知识。(1)研究了高维约束正则化的统一框架，并进一步研究了高维数据的一般框架在折叠凹罚和固定约束下的统计性能。(2)探索了高效的分布式受限统计学习算法，研究了通信高效的样本分裂算法，用于处理具有折叠凹罚的高维约束正则化问题中的大量样本和高通信代价。该项目将研究非凸学习的新算法的收敛以及统计收敛。这项工作论证了同时考虑统计分析和优化分析的必要性。(3)研究人员计划将该方法应用于分析大数据集，以解决与帕金森病相关的临床和科学问题。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

In the era of big data, data sets may be massive in size and have high dimensionality. Examples include social media data, high-resolution image data, and genomic data. High dimensionality and massive sample size pose great computational and statistical challenges. This project aims to investigate novel optimization techniques and statistical analytic tools for big data collected in important applications. The project will significantly enhance the capabilities of optimization and statistics in analyzing big data. Results of the project are anticipated to benefit a broad range of areas including public health, medical studies, and financial portfolio management.This project consists of three sub-projects to advance knowledge in modern optimization techniques and statistical procedures for big data. (1) The project studies a unified framework for high-dimensional constrained regularization, and further investigates the statistical performance of a general framework for high-dimensional data under folded concave penalty and fixed constraints. (2) The project explores efficient distributed algorithms for constrained statistical learning, investigating communication-efficient sample-splitting algorithms to handle the vast number of samples and high communication cost in high-dimensional constrained regularization problems with folded concave penalty. The project will study the convergence of the new algorithms for non-convex learning as well as statistical convergence. The work demonstrates the necessity to consider statistical analysis and optimization analysis simultaneously. (3) The investigators plan to apply the methodology in analyzing big data sets to address clinical and scientific questions related to Parkinson's disease.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.