BIGDATA: Collaborative Research: F: Foundations of Nonconvex Problems in BigData Science and Engineering: Models, Algorithms, and Analysis

BIGDATA：协作研究：F：大数据科学与工程中非凸问题的基础：模型、算法和分析

• 批准号: 1632935
• 负责人: Jack Xin
• 金额: $ 35万
• 依托单位: University of California-Irvine
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-01 至 2020-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1632935&HistoricalAwards=false
• 关键词: BIGDATA; Collaborative; Research; Foundations; Nonconvex

In today's digital world, huge amounts of data, i.e., big data, can be found in almost every aspect of scientific research and human activity. These data need to be managed effectively for reliable prediction and inference to improve decision making. Statistical learning is an emergent scientific discipline wherein mathematical modeling, computational algorithms, and statistical analysis are jointly employed to address these challenging data management problems. Invariably, quantitative criteria need to be introduced for the overall learning process in order to gauge the quality of the solutions obtained. This research focuses on two important criteria: data fitness and sparsity representation of the underlying learning model. Potential applications of the results can be found in computational statistics, compressed sensing, imaging, machine learning, bio-informatics, portfolio selection, and decision making under uncertainty, among many areas involving big data.Till now, convex optimization has been the dominant methodology for statistical learning in which the two criteria employed are expressed by convex functions either to be optimized and/or set as constraints of the variables being sought. Recently, non-convex functions of the difference-of-convex (DC) type and the difference-of-convex algorithm (DCA) have been shown to yield superior results in many contexts and serve as the motivation for this project. The goal is to develop a solid foundation and a unified framework to address many fundamental issues in big data problems in which non-convexity and non-differentiability are present in the optimization problems to be solved. These two non-standard features in computational statistical learning are challenging and their rigorous treatment requires the fusion of expertise from different domains of mathematical sciences. Technical issues to be investigated will cover the optimality, sparsity, and statistical properties of computable solutions to the non-convex, non-smooth optimization problems arising from statistical learning and its many applications. Novel algorithms will be developed and tested first on synthetic data sets for preliminary experimentation and then on publicly available data sets for realism; comparisons will be made among different formulations of the learning problems.

在当今的数字世界中，大量的数据，即，大数据几乎可以在科学研究和人类活动的各个方面找到。 需要有效管理这些数据，以进行可靠的预测和推理，从而改善决策制定。 统计学习是一门新兴的科学学科，其中数学建模，计算算法和统计分析联合使用，以解决这些具有挑战性的数据管理问题。 因此，需要为整个学习过程引入量化标准，以衡量所获得解决方案的质量。本研究着重于两个重要的标准：数据适应度和底层学习模型的稀疏表示。 在涉及大数据的许多领域中，可以在计算统计、压缩感知、成像、机器学习、生物信息学、投资组合选择和不确定性下的决策中找到结果的潜在应用。凸优化已经成为统计学习的主要方法，其中所采用的两个标准由凸函数来表示，或者被优化，或者/或者被设置为所寻找的变量的约束。 最近，非凸函数的差分凸（DC）型和差分凸算法（DCA）已被证明在许多情况下产生上级的结果，并作为本项目的动机。 目标是开发一个坚实的基础和统一的框架，以解决大数据问题中的许多基本问题，其中非凸性和不可微性存在于待解决的优化问题中。计算统计学习中的这两个非标准特征是具有挑战性的，它们的严格处理需要融合来自不同数学科学领域的专业知识。 要调查的技术问题将涵盖最优性，稀疏性和统计特性的可计算解决方案的非凸，非光滑的优化问题所产生的统计学习及其许多应用。 新的算法将首先在合成数据集上进行初步实验和测试，然后在公开可用的数据集上进行现实主义;将在学习问题的不同配方之间进行比较。

项目成果

BinaryRelax: A Relaxation Approach for Training Deep Neural Networks with Quantized Weights

• DOI: 10.1137/18m1166134
• 发表时间: 2018-01-01
• 期刊: SIAM JOURNAL ON IMAGING SCIENCES
• 影响因子: 2.1
• 作者: Yin, Penghang;Zhang, Shuai;Xin, Jack
• 通讯作者: Xin, Jack

Blended coarse gradient descent for full quantization of deep neural networks

• DOI: 10.1007/s40687-018-0177-6
• 发表时间: 2018-08
• 期刊: Research in the Mathematical Sciences
• 影响因子: 1.2
• 作者: Penghang Yin;Shuai Zhang;J. Lyu;S. Osher;Y. Qi;J. Xin

Learning Sparse Neural Networks via ℓ0 and Tℓ1 by a Relaxed Variable Splitting Method with Application to Multi-scale Curve Classification

通过松弛变量分裂方法通过α0和Tα1学习稀疏神经网络并应用于多尺度曲线分类

• DOI: 10.1007/978-3-030-21803-4
• 发表时间: 2020
• 期刊: 6th World Congress on Global Optimization
• 作者: Xue, F;Xin, J
• 通讯作者: Xin, J

Computing Residual Diffusivity by Adaptive Basis Learning via Super-Resolution Deep Neural Networks

通过超分辨率深度神经网络的自适应基础学习计算残余扩散率

• DOI: 10.1007/978-3-030-38364-0_25
• 发表时间: 2020
• 期刊: Applied Mathematics and Applications
• 作者: Lyu, Jiancheng;Xin, Jack;Yu, Yifeng
• 通讯作者: Yu, Yifeng

Convergence of a Relaxed Variable Splitting Method for Learning Sparse Neural Networks via L1, L0, and transformed-L1 Penalties

通过 L1、L0 和变换 L1 惩罚学习稀疏神经网络的宽松变量分裂方法的收敛性

• 发表时间: 2020
• 期刊: Intelligent Systems Conference (IntelliSys
• 作者: Dinh, Thu;Xin, Jack
• 通讯作者: Xin, Jack