CAREER: Scaling Up Knowledge Discovery in High-Dimensional Data Via Nonconvex Statistical Optimization

职业：通过非凸统计优化扩大高维数据中的知识发现

• 批准号: 1652539
• 负责人: Quanquan Gu
• 金额: $ 51.58万
• 依托单位: University of Virginia Main Campus
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-08-01 至 2018-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1652539&HistoricalAwards=false
• 关键词: CAREER; Scaling; Up; Knowledge; Discovery

The past decade has witnessed a surge of research activities on knowledge discovery in high-dimensional data, among which convex optimization-based methods are widely used. While convex optimization algorithms enjoy global convergence guarantees, they are not always scalable to high-dimensional massive data. Motivated by the empirical success of nonconvex methods such as matrix factorization, the objective of this project is to develop a new generation of principled nonconvex statistical optimization algorithms to scale up high-dimensional machine learning methods. This project amplifies the utility of high-dimensional knowledge discovery methods in various fields such as computational genomics and recommendation systems. It incorporates the resulting research outcomes into curriculum development and online courses, to train a new generation of machine learning and data mining practitioners. In addition, special training is provided to K-12 students and community college students for a broader education of modern data analysis techniques.This project consists of three synergistic research thrusts. First, it develops a family of nonconvex algorithms for structured sparse learning, including extensions to both parallel computing and distributed computing. Second, it devises a unified nonconvex optimization framework for low-rank matrix estimation, which covers a wide range of low-rank matrix learning problems such as matrix completion and preference learning. Several acceleration techniques are also explored. Third, it develops a family of alternating optimization algorithms, to solve the bi-convex optimization problem for estimating various complex statistical models. This project integrates modern optimization techniques with model-based statistical thinking, and provides a systematic way to design nonconvex high-dimensional machine learning methods with strong theoretical guarantees. The targeted applications include but not limited to computational genomics, neuroscience, and recommendation systems.

在过去的十年中，高维数据中的知识发现的研究活动激增，其中，基于凸优化的方法被广泛使用。虽然凸优化算法具有全局收敛性保证，但它们并不总是可扩展到高维海量数据。受到矩阵分解等非凸方法的经验成功的启发，该项目的目标是开发新一代有原则的非凸统计优化算法，以扩展高维机器学习方法。该项目放大了高维知识发现方法在计算基因组学和推荐系统等各个领域的实用性。它将由此产生的研究成果纳入课程开发和在线课程，以培养新一代机器学习和数据挖掘从业人员。此外，还为K-12学生和社区大学学生提供特别培训，以更广泛地教育现代数据分析技术。首先，它开发了一个家庭的非凸算法的结构化稀疏学习，包括并行计算和分布式计算的扩展。其次，它设计了一个统一的低秩矩阵估计的非凸优化框架，它涵盖了广泛的低秩矩阵学习问题，如矩阵完成和偏好学习。还探讨了几种加速技术。第三，发展了一系列交替优化算法，用于求解各种复杂统计模型的双凸优化问题。该项目将现代优化技术与基于模型的统计思想相结合，为设计非凸高维机器学习方法提供了一种系统的方法，具有强有力的理论保证。目标应用包括但不限于计算基因组学、神经科学和推荐系统。

项目成果

Accelerated Stochastic Mirror Descent: From Continuous-time Dynamics to Discrete-time Algorithms

加速随机镜像下降：从连续时间动力学到离散时间算法

• 发表时间: 2018
• 期刊: International Conference on Artificial Intelligence and Statistics
• 作者: Xu, Pan;Wang, Tianhao;Gu, Quanquan
• 通讯作者: Gu, Quanquan

A Unified Framework for Nonconvex Low-Rank plus Sparse Matrix Recovery

• 发表时间: 2018-03
• 作者: Xiao Zhang;Lingxiao Wang;Quanquan Gu

Stochastic Variance-Reduced Hamilton Monte Carlo Methods

• 发表时间: 2018-02
• 期刊: ArXiv
• 作者: Difan Zou;Pan Xu;Quanquan Gu

Stochastic Variance-Reduced Cubic Regularized Newton Method

• 发表时间: 2018-02
• 期刊: ArXiv
• 作者: Dongruo Zhou;Pan Xu;Quanquan Gu

Stochastic Nested Variance Reduced Gradient Descent for Nonconvex Optimization

• 发表时间: 2018
• 作者: Dongruo Zhou;Pan Xu;Quanquan Gu