CAREER: Scaling Up Knowledge Discovery in High-Dimensional Data Via Nonconvex Statistical Optimization

近十年来，高维数据的知识发现研究兴起，其中基于凸优化的方法得到了广泛的应用。虽然凸优化算法具有全局收敛性保证，但它们并不总是可扩展到高维海量数据。受非凸方法（如矩阵分解）的经验成功的激励，本项目的目标是开发新一代有原则的非凸统计优化算法，以扩展高维机器学习方法。该项目扩大了高维知识发现方法在计算基因组学和推荐系统等各个领域的效用。它将由此产生的研究成果纳入课程开发和在线课程，以培养新一代机器学习和数据挖掘从业者。此外，为K-12学生和社区大学学生提供特殊培训，以接受更广泛的现代数据分析技术教育。该项目包括三个协同研究重点。首先，它开发了一系列用于结构化稀疏学习的非凸算法，包括对并行计算和分布式计算的扩展。其次，设计了一个统一的低秩矩阵估计非凸优化框架，涵盖了矩阵补全、偏好学习等广泛的低秩矩阵学习问题；还探讨了几种加速技术。第三，开发了一组交替优化算法，解决了估计各种复杂统计模型的双凸优化问题。本项目将现代优化技术与基于模型的统计思维相结合，为非凸高维机器学习方法的设计提供了一种系统的方法，具有较强的理论保障。目标应用包括但不限于计算基因组学、神经科学和推荐系统。

The past decade has witnessed a surge of research activities on knowledge discovery in high-dimensional data, among which convex optimization-based methods are widely used. While convex optimization algorithms enjoy global convergence guarantees, they are not always scalable to high-dimensional massive data. Motivated by the empirical success of nonconvex methods such as matrix factorization, the objective of this project is to develop a new generation of principled nonconvex statistical optimization algorithms to scale up high-dimensional machine learning methods. This project amplifies the utility of high-dimensional knowledge discovery methods in various fields such as computational genomics and recommendation systems. It incorporates the resulting research outcomes into curriculum development and online courses, to train a new generation of machine learning and data mining practitioners. In addition, special training is provided to K-12 students and community college students for a broader education of modern data analysis techniques.This project consists of three synergistic research thrusts. First, it develops a family of nonconvex algorithms for structured sparse learning, including extensions to both parallel computing and distributed computing. Second, it devises a unified nonconvex optimization framework for low-rank matrix estimation, which covers a wide range of low-rank matrix learning problems such as matrix completion and preference learning. Several acceleration techniques are also explored. Third, it develops a family of alternating optimization algorithms, to solve the bi-convex optimization problem for estimating various complex statistical models. This project integrates modern optimization techniques with model-based statistical thinking, and provides a systematic way to design nonconvex high-dimensional machine learning methods with strong theoretical guarantees. The targeted applications include but not limited to computational genomics, neuroscience, and recommendation systems.