Bundle Level Type Gradient Sliding Methods for Large Scale Convex Optimization

这项研究的目标是开发新的算法，以解决在分析具有巨大数据集的应用程序的数据时所涉及的计算挑战。这些包括，例如，图像处理，数据挖掘，生物信息学和统计学习。在这项研究中开发的算法将能够显着减少所需的昂贵计算的数量，使它们可以应用于有效地从海量数据集中提取有用的信息。该研究有可能推进大规模问题的算法，并大大增加许多新兴技术的适用性。一个例子是使用部分并行磁共振成像采集的图像的有效重建。新方法的开发还将使研究人员能够构建多级复杂网络，以便在许多应用中进行更好的学习和预测。该项目还通过本科生和研究生培训、课程开发以及研讨会和会议演示来支持教育。本研究旨在发展一类新的求解大规模复合凸优化问题和泛函约束凸优化问题的加速束水平型梯度滑动算法及其相关理论。这类新的算法预计将分别为每个组件实现最佳迭代复杂度，但将更加通用，能够处理具有不同平滑度的函数的组合。该算法提供的优点是有效地利用历史信息，有一个可扩展的方案来解决所涉及的子问题，提供实际的终止条件的梯度滑动，并不施加限制的步长或要求的信息的Lipschitz常数的成本函数。此外，这些技术的发展功能约束问题将显着降低迭代的复杂性，并提高现有的技术的实际性能的函数是光滑或弱光滑。此外，复合梯度滑动和加速的方法减少了梯度评估的数量，而不增加迭代的复杂性，同时保持现有的良好性能的复合凸问题的方法。将分析所有新算法的迭代复杂度，并通过数值模拟以及成像和机器学习的实际应用验证实际性能。

The goal of this research is to develop novel algorithms for tackling the computational challenges involved in analyzing data for applications with huge data sets. These include, for example, image processing, data mining, bioinformatics, and statistical learning. The algorithms to be developed in this research will be able to significantly reduce the number of required expensive computations, so that they can be applied to efficiently extract useful information from massive data sets. The research has the potential to advance the algorithms for large scale problems, and greatly increase the applicability for many emerging technologies. An example is the efficient reconstruction of images acquired using partial parallel magnetic resonance imaging. The development of the new methods will also enable researchers to build multi-level complex networks for better learning and prediction in many applications. This project also supports education through undergraduate and graduate student training, course development, and seminar and conference presentations. This research intends to develop a novel class of accelerated bundle level type gradient sliding methods and related theories for solving large scale composite convex optimization problems and functional constrained convex optimization problems. This new class of algorithms is expected to achieve optimal iteration complexity for each component separately, but will be more general and able to handle the composition of functions with various degrees of smoothness. The algorithms offer the advantages of effectively using historical information, having a scalable scheme for solving the involved sub-problem, providing practical termination conditions for the gradient sliding, and do not impose restrictions on step sizes or require the information on the Lipschitz constants in the cost functions. Moreover, the development of these techniques for the functionally constrained problems will significantly reduce the iteration complexities and improve the practical performance of the existing techniques for functions that are smooth or weakly smooth. Further, the composite gradient sliding and accelerated approach reduces the number of gradient evaluations without increasing the iteration complexity, while maintaining existing good properties of the approaches for the composite convex problems. The iteration complexity of all the new algorithms will be analyzed, and the practical performance will be validated through numerical simulations and for practical applications arising from imaging and machine learning.