BIGDATA: Collaborative Research: F: Foundations of Nonconvex Problems in BigData Science and Engineering: Models, Algorithms, and Analysis

在当今的数字世界中，大量的数据，即，大数据几乎可以在科学研究和人类活动的各个方面找到。 这些数据需要有效管理，以进行可靠的预测和推理，从而改善决策。 统计学习是一门新兴的科学学科，其中数学建模，计算算法和统计分析联合使用，以解决这些具有挑战性的数据管理问题。 因此，需要为整个学习过程引入量化标准，以衡量所获得解决方案的质量。本研究着重于两个重要的标准：数据适应度和底层学习模型的稀疏表示。 在涉及大数据的许多领域中，可以在计算统计、压缩感知、成像、机器学习、生物信息学、投资组合选择和不确定性下的决策中找到结果的潜在应用。凸优化已经成为统计学习的主要方法，其中所采用的两个标准由凸函数来表示，或者被优化，或者/或者被设置为所寻找的变量的约束。 最近，非凸函数的差分凸（DC）型和差分凸算法（DCA）已被证明在许多情况下产生上级的结果，并作为本项目的动机。 目标是开发一个坚实的基础和统一的框架，以解决大数据问题中的许多基本问题，其中非凸性和不可微性存在于待解决的优化问题中。计算统计学习中的这两个非标准特征是具有挑战性的，它们的严格处理需要融合来自不同数学科学领域的专业知识。 要调查的技术问题将涵盖最优性，稀疏性和统计特性的可计算解决方案的非凸，非光滑的优化问题所产生的统计学习及其许多应用。 新的算法将首先在合成数据集上进行初步实验和测试，然后在公开的数据集上进行真实性测试;将在学习问题的不同配方之间进行比较。

In today's digital world, huge amounts of data, i.e., big data, can be found in almost every aspect of scientific research and human activity. These data need to be managed effectively for reliable prediction and inference to improve decision making. Statistical learning is an emergent scientific discipline wherein mathematical modeling, computational algorithms, and statistical analysis are jointly employed to address these challenging data management problems. Invariably, quantitative criteria need to be introduced for the overall learning process in order to gauge the quality of the solutions obtained. This research focuses on two important criteria: data fitness and sparsity representation of the underlying learning model. Potential applications of the results can be found in computational statistics, compressed sensing, imaging, machine learning, bio-informatics, portfolio selection, and decision making under uncertainty, among many areas involving big data.Till now, convex optimization has been the dominant methodology for statistical learning in which the two criteria employed are expressed by convex functions either to be optimized and/or set as constraints of the variables being sought. Recently, non-convex functions of the difference-of-convex (DC) type and the difference-of-convex algorithm (DCA) have been shown to yield superior results in many contexts and serve as the motivation for this project. The goal is to develop a solid foundation and a unified framework to address many fundamental issues in big data problems in which non-convexity and non-differentiability are present in the optimization problems to be solved. These two non-standard features in computational statistical learning are challenging and their rigorous treatment requires the fusion of expertise from different domains of mathematical sciences. Technical issues to be investigated will cover the optimality, sparsity, and statistical properties of computable solutions to the non-convex, non-smooth optimization problems arising from statistical learning and its many applications. Novel algorithms will be developed and tested first on synthetic data sets for preliminary experimentation and then on publicly available data sets for realism; comparisons will be made among different formulations of the learning problems.