BIGDATA: Collaborative Research: F: Foundations of Nonconvex Problems in BigData Science and Engineering: Models, Algorithms, and Analysis

BIGDATA：协作研究：F：大数据科学与工程中非凸问题的基础：模型、算法和分析

• 批准号: 1632935
• 负责人: Jack Xin
• 金额: $ 35万
• 依托单位: University of California-Irvine
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-01 至 2020-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1632935&HistoricalAwards=false
• 关键词: BIGDATA; Collaborative; Research; Foundations; Nonconvex

In today's digital world, huge amounts of data, i.e., big data, can be found in almost every aspect of scientific research and human activity. These data need to be managed effectively for reliable prediction and inference to improve decision making. Statistical learning is an emergent scientific discipline wherein mathematical modeling, computational algorithms, and statistical analysis are jointly employed to address these challenging data management problems. Invariably, quantitative criteria need to be introduced for the overall learning process in order to gauge the quality of the solutions obtained. This research focuses on two important criteria: data fitness and sparsity representation of the underlying learning model. Potential applications of the results can be found in computational statistics, compressed sensing, imaging, machine learning, bio-informatics, portfolio selection, and decision making under uncertainty, among many areas involving big data.Till now, convex optimization has been the dominant methodology for statistical learning in which the two criteria employed are expressed by convex functions either to be optimized and/or set as constraints of the variables being sought. Recently, non-convex functions of the difference-of-convex (DC) type and the difference-of-convex algorithm (DCA) have been shown to yield superior results in many contexts and serve as the motivation for this project. The goal is to develop a solid foundation and a unified framework to address many fundamental issues in big data problems in which non-convexity and non-differentiability are present in the optimization problems to be solved. These two non-standard features in computational statistical learning are challenging and their rigorous treatment requires the fusion of expertise from different domains of mathematical sciences. Technical issues to be investigated will cover the optimality, sparsity, and statistical properties of computable solutions to the non-convex, non-smooth optimization problems arising from statistical learning and its many applications. Novel algorithms will be developed and tested first on synthetic data sets for preliminary experimentation and then on publicly available data sets for realism; comparisons will be made among different formulations of the learning problems.

在当今的数字世界中，在科学研究和人类活动的几乎每个方面都可以找到大量数据，即大数据。 这些数据需要有效地管理，以进行可靠的预测和推理，以改善决策。 统计学习是一门新兴的科学学科，在其中共同使用数学建模，计算算法和统计分析来解决这些具有挑战性的数据管理问题。 为了评估获得的解决方案的质量，需要引入整体学习过程的定量标准。这项研究重点介绍了两个重要标准：基础学习模型的数据适应性和稀疏性表示。 结果的潜在应用可以在计算统计，压缩感应，成像，机器学习，生物信息学，投资组合选择和不确定性下的决策中，在许多涉及大数据的领域中。 最近，已经证明，偶数差异（DC）类型（DC）类型的非凸功能功能已证明在许多情况下可以在许多情况下产生出色的结果，并作为该项目的动机。 目的是建立坚实的基础和一个统一的框架，以解决大数据问题中的许多基本问题，在这些问题中，在要解决的优化问题中存在非跨性别和非差异性。计算统计学习中的这两个非标准特征具有挑战性，它们的严格治疗需要从数学科学的不同领域融合专业知识。 要调查的技术问题将涵盖可计算解决方案对非凸的可计算解决方案的最佳性，稀疏性和统计特性，这是由于统计学习及其许多应用而引起的非平滑优化问题。 新颖的算法将首先在初步实验的合成数据集上开发和测试，然后在现实主义的公开数据集上进行。将在学习问题的不同表述中进行比较。

项目成果

Blended coarse gradient descent for full quantization of deep neural networks

• DOI: 10.1007/s40687-018-0177-6
• 发表时间: 2018-08
• 期刊: Research in the Mathematical Sciences
• 影响因子: 1.2
• 作者: Penghang Yin;Shuai Zhang;J. Lyu;S. Osher;Y. Qi;J. Xin

BinaryRelax: A Relaxation Approach for Training Deep Neural Networks with Quantized Weights

• DOI: 10.1137/18m1166134
• 发表时间: 2018-01-01
• 期刊: SIAM JOURNAL ON IMAGING SCIENCES
• 影响因子: 2.1
• 作者: Yin, Penghang;Zhang, Shuai;Xin, Jack
• 通讯作者: Xin, Jack

Learning Sparse Neural Networks via ℓ0 and Tℓ1 by a Relaxed Variable Splitting Method with Application to Multi-scale Curve Classification

通过松弛变量分裂方法通过α0和Tα1学习稀疏神经网络并应用于多尺度曲线分类

• DOI: 10.1007/978-3-030-21803-4
• 发表时间: 2020
• 期刊: 6th World Congress on Global Optimization
• 作者: Xue, F;Xin, J
• 通讯作者: Xin, J

Computing Residual Diffusivity by Adaptive Basis Learning via Super-Resolution Deep Neural Networks

通过超分辨率深度神经网络的自适应基础学习计算残余扩散率

• DOI: 10.1007/978-3-030-38364-0_25
• 发表时间: 2020
• 期刊: Applied Mathematics and Applications
• 作者: Lyu, Jiancheng;Xin, Jack;Yu, Yifeng
• 通讯作者: Yu, Yifeng

Convergence of a Relaxed Variable Splitting Method for Learning Sparse Neural Networks via L1, L0, and transformed-L1 Penalties

通过 L1、L0 和变换 L1 惩罚学习稀疏神经网络的宽松变量分裂方法的收敛性

• 发表时间: 2020
• 期刊: Intelligent Systems Conference (IntelliSys
• 作者: Dinh, Thu;Xin, Jack
• 通讯作者: Xin, Jack