CAREER: Scaling Up Knowledge Discovery in High-Dimensional Data Via Nonconvex Statistical Optimization

在过去的十年里，高维数据中的知识发现研究活动激增，其中基于凸优化的方法得到了广泛的应用。虽然凸优化算法享有全局收敛的保证，但它们并不总是可扩展到高维海量数据。受矩阵分解等非凸方法的成功经验的启发，本项目的目标是开发新一代原则性非凸统计优化算法，以扩大高维机器学习方法的规模。该项目扩大了高维知识发现方法在各种领域的应用，如计算基因组学和推荐系统。它将由此产生的研究成果纳入课程开发和在线课程，以培训新一代机器学习和数据挖掘从业者。此外，还为K-12学生和社区学院的学生提供了专门的培训，使他们能够更广泛地学习现代数据分析技术。首先，它开发了一系列用于结构化稀疏学习的非凸算法，包括对并行计算和分布式计算的扩展。其次，设计了低阶矩阵估计的统一非凸优化框架，该框架涵盖了矩阵补全和偏好学习等广泛的低阶矩阵学习问题。文中还探讨了几种加速技术。第三，开发了一类交替优化算法，用于解决估计各种复杂统计模型的双凸优化问题。该项目将现代优化技术与基于模型的统计思想相结合，为非凸高维机器学习方法的设计提供了一种系统的方法，具有较强的理论保障。目标应用包括但不限于计算基因组学、神经科学和推荐系统。

The past decade has witnessed a surge of research activities on knowledge discovery in high-dimensional data, among which convex optimization-based methods are widely used. While convex optimization algorithms enjoy global convergence guarantees, they are not always scalable to high-dimensional massive data. Motivated by the empirical success of nonconvex methods such as matrix factorization, the objective of this project is to develop a new generation of principled nonconvex statistical optimization algorithms to scale up high-dimensional machine learning methods. This project amplifies the utility of high-dimensional knowledge discovery methods in various fields such as computational genomics and recommendation systems. It incorporates the resulting research outcomes into curriculum development and online courses, to train a new generation of machine learning and data mining practitioners. In addition, special training is provided to K-12 students and community college students for a broader education of modern data analysis techniques.This project consists of three synergistic research thrusts. First, it develops a family of nonconvex algorithms for structured sparse learning, including extensions to both parallel computing and distributed computing. Second, it devises a unified nonconvex optimization framework for low-rank matrix estimation, which covers a wide range of low-rank matrix learning problems such as matrix completion and preference learning. Several acceleration techniques are also explored. Third, it develops a family of alternating optimization algorithms, to solve the bi-convex optimization problem for estimating various complex statistical models. This project integrates modern optimization techniques with model-based statistical thinking, and provides a systematic way to design nonconvex high-dimensional machine learning methods with strong theoretical guarantees. The targeted applications include but not limited to computational genomics, neuroscience, and recommendation systems.