RI: Medium: Collaborative Research: Next-Generation Statistical Optimization Methods for Big Data Computing

该项目开发了新一代优化方法，以解决大规模科学数据分析中的数据挖掘和知识发现挑战。该项目是在现代计算架构使我们能够在大型和复杂的数据集（大数据）上拟合复杂的统计模型（大模型）的背景下构建的。然而，尽管在大数据、大模型和现代计算架构的各个子领域都取得了重大进展，但我们仍然缺乏强大的优化技术来有效地整合这些关键组件。一个重要的瓶颈是，许多通用优化方法不是专门为统计学习问题设计的。即使其中一些是为利用特定的问题结构而量身定制的，它们实际上并没有将复杂的统计思维融入算法设计和分析中。为了解决这一瓶颈，该项目扩展了传统理论，为非传统优化问题（如非凸和无限维示例）开辟了新的可能性。该项目对优化中的几个具有挑战性的问题（如非凸性）进行了更深入的理论理解，开发了新的算法，将在大数据时代带来更好的实用方法，并在具有挑战性的生物信息学问题上展示了新方法。该项目与NSF推动大数据研究的使命密切相关，将产生广泛的影响。在大数据时代，我们迫切需要强大的优化方法来处理日益复杂的现代数据集。然而，我们仍然缺乏足够的方法、理论和计算技术。通过同时解决这些问题，该项目将提供新颖而有用的统计优化方法，使所有相关科学领域受益。该项目将提供易于使用的软件包，直接帮助科学家探索和分析复杂的数据集。这两个pi还将设计和开发新的课程，教授处理大数据优化问题的现代技术。所有的课程材料——包括课堂讲稿、习题集、源代码、解决方案和工作示例——都将在网上免费获取。此外，两位pi将撰写指导论文，并通过互联网、学术会议、研讨会和期刊传播这项研究的结果。通过高级论文和潜在的REU（本科生研究经验）计划，拟议的项目也将积极包括本科生和参与代表性不足的少数群体。为了实现这些目标，本项目开发了(1)一个名为统计优化的新研究领域，它将复杂的统计思维融入现代优化中，并将有效地连接机器学习、统计、优化和随机分析；（ii）新的非凸和无限维优化的理论框架和计算方法，这将激发有效的优化方法与理论保证，适用于各种突出的统计模型；（iii）新的可扩展优化方法，旨在充分利用现代大规模分布式计算基础设施的马力。该项目将为大规模优化提供新的理论依据，通过新颖的算法和软件推进实践，并展示具有挑战性的生物信息学问题的方法。

This project develops a new generation of optimization methods to address data mining and knowledge discovery challenges in large-scale scientific data analysis. The project is constructed in the context that modern computing architectures are enabling us to fit complex statistical models (Big Models) on large and complex datasets (Big Data). However, despite significant progress in each subfield of Big Data, Big Model, and modern computing architecture, we are still lacking powerful optimization techniques to effectively integrate these key components.One important bottleneck is that many general-purpose optimization methods are not specifically designed for statistical learning problems. Even some of them are tailored to utilize specific problem structures, they have not actually incorporated sophisticated statistical thinking into algorithm design and analysis. To tackle this bottleneck, the project extends traditional theory to open new possibilities for nontraditional optimization problems, such as nonconvex and infinite-dimensional examples. The project develops deeper theoretical understanding of several challenging issues in optimization (such as nonconvexity), develops new algorithms that will lead to better practical methods in the big data era, and demonstrates the new methods on challenging bio-informatics problems.The project is closely related to NSF's mission to promote Big Data research, and will have broad impacts. In the Big Data era, we see an urgent need for powerful optimization methods to handle the increasing complexity of modern datasets. However, we still lack adequate methods, theory, and computational techniques. By simultaneously addressing these aspects, this project will deliver novel and useful statistical optimization methods that benefit all relevant scientific areas. The project will deliver easy-to-use software packages which directly help scientists to explore and analyze complex datasets. Both PIs will also design and develop new classes to teach modern techniques in handling big data optimization problems. All the course materials - including lecture notes, problem sets, source code, solutions and working examples - will be freely accessed online. Moreover, both PIs will write tutorial papers and disseminate the results of this research through the internet, academic conferences, workshops, and journals. Through senior theses and potentially the REU (Research Experiences for Undergraduates) program, the proposed project will also actively include undergraduates and engage under-represented minority groups.To achieve these goals, this project develops (i) a new research area named statistical optimization, which incorporates sophisticated statistical thinking into modern optimization, and will effectively bridge machine learning, statistics, optimization, and stochastic analysis; (ii) new theoretical frameworks and computational methods for nonconvex and infinite-dimensional optimization, which will motivate effective optimization methods with theoretical guarantees that are applicable to a wide variety of prominent statistical models; (iii) new scalable optimization methods, which aim at fully harnessing the horsepower of modern large-scale distributed computing infrastructure. The project will shed new theoretical light on large-scale optimization, advance practice through novel algorithms and software, and demonstrate the methods on challenging bio-informatics problems.