CAREER: Scaling Up Knowledge Discovery in High-Dimensional Data Via Nonconvex Statistical Optimization

职业：通过非凸统计优化扩大高维数据中的知识发现

• 批准号: 1906169
• 负责人: Quanquan Gu
• 金额: $ 50.6万
• 依托单位: University of California-Los Angeles
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-07-01 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1906169&HistoricalAwards=false
• 关键词: CAREER; Scaling; Up; Knowledge; Discovery

The past decade has witnessed a surge of research activities on knowledge discovery in high-dimensional data, among which convex optimization-based methods are widely used. While convex optimization algorithms enjoy global convergence guarantees, they are not always scalable to high-dimensional massive data. Motivated by the empirical success of nonconvex methods such as matrix factorization, the objective of this project is to develop a new generation of principled nonconvex statistical optimization algorithms to scale up high-dimensional machine learning methods. This project amplifies the utility of high-dimensional knowledge discovery methods in various fields such as computational genomics and recommendation systems. It incorporates the resulting research outcomes into curriculum development and online courses, to train a new generation of machine learning and data mining practitioners. In addition, special training is provided to K-12 students and community college students for a broader education of modern data analysis techniques.This project consists of three synergistic research thrusts. First, it develops a family of nonconvex algorithms for structured sparse learning, including extensions to both parallel computing and distributed computing. Second, it devises a unified nonconvex optimization framework for low-rank matrix estimation, which covers a wide range of low-rank matrix learning problems such as matrix completion and preference learning. Several acceleration techniques are also explored. Third, it develops a family of alternating optimization algorithms, to solve the bi-convex optimization problem for estimating various complex statistical models. This project integrates modern optimization techniques with model-based statistical thinking, and provides a systematic way to design nonconvex high-dimensional machine learning methods with strong theoretical guarantees. The targeted applications include but not limited to computational genomics, neuroscience, and recommendation systems.

在过去的十年中，高维数据中的知识发现的研究活动激增，其中，基于凸优化的方法被广泛使用。虽然凸优化算法具有全局收敛性保证，但它们并不总是可扩展到高维海量数据。受矩阵分解等非凸方法的经验成功的启发，该项目的目标是开发新一代原则性非凸统计优化算法，以扩展高维机器学习方法。该项目放大了高维知识发现方法在计算基因组学和推荐系统等各个领域的实用性。它将由此产生的研究成果纳入课程开发和在线课程，以培养新一代机器学习和数据挖掘从业人员。此外，还为K-12学生和社区大学学生提供特别培训，以更广泛地教育现代数据分析技术。首先，它开发了一个家庭的非凸算法的结构化稀疏学习，包括并行计算和分布式计算的扩展。其次，它设计了一个统一的低秩矩阵估计的非凸优化框架，它涵盖了广泛的低秩矩阵学习问题，如矩阵完成和偏好学习。还探讨了几种加速技术。第三，发展了一系列交替优化算法，用于求解各种复杂统计模型的双凸优化问题。该项目将现代优化技术与基于模型的统计思想相结合，为设计非凸高维机器学习方法提供了一种系统的方法，具有强有力的理论保证。目标应用包括但不限于计算基因组学、神经科学和推荐系统。

项目成果

A Simple and Provably Efficient Algorithm for Asynchronous Federated Contextual Linear Bandits

• DOI: 10.48550/arxiv.2207.03106
• 发表时间: 2022-07
• 期刊: ArXiv
• 作者: Jiafan He;Tianhao Wang;Yifei Min;Quanquan Gu

Accelerated Factored Gradient Descent for Low-Rank Matrix Factorization

• 发表时间: 2020
• 作者: Dongruo Zhou;Yuan Cao;Quanquan Gu

On the Global Convergence of Training Deep Linear ResNets

• 发表时间: 2020-03
• 期刊: ArXiv
• 作者: Difan Zou;Philip M. Long;Quanquan Gu

Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks

• 发表时间: 2019-11
• 期刊: ArXiv
• 作者: Difan Zou;Ziniu Hu;Yewen Wang;Song Jiang;Yizhou Sun;Quanquan Gu

Last Iterate Risk Bounds of SGD with Decaying Stepsize for Overparameterized Linear Regression

超参数化线性回归的衰减步长 SGD 的最后迭代风险界限

• 发表时间: 2022
• 期刊: Proceedings of Machine Learning Research
• 作者: Wu, Jingfeng;Zou, Difan;Braverman, Vladimir;Gu, Quanquan;Kakade, Sham
• 通讯作者: Kakade, Sham