CAREER: Theory and Methods for Simultaneous Variable Selection and Rank Reduction

在所有科学领域的数据爆炸创造了一个迫切需要的方法来分析高维多变量数据。该项目深化和拓展了现有的稀疏性和低秩统计理论和方法，取得了以下主要科学成果：（a）通过同时进行变量选择和投影的创新性可选降秩方法，保证了理论上比现有变量选择和降秩率更低的错误率，为高维统计和信息论的新前沿铺平了道路;（B）快速但易于实现的算法，可以处理所有流行的惩罚函数（可能是非凸的）在计算中具有保证的全局收敛性和局部最优性，以确保所提出的方法在大数据应用中的实用性;（c）对能够考虑多变量响应之间的相关性的非高斯模型的一般扩展，（d）统一的稳健化方案，既能识别又能适应真实的数据中经常出现的总体异常值，以克服许多传统多变量工具的不稳健性;（e）用于变量选择和/或降秩的通用模型选择方法，在理论保证下实现有限样本最优预测误差率。 从高维多变量噪声数据中恢复低维信号的需求渗透到科学和工程的各个领域。因此，这种性质的项目，旨在开发变革性的理论和方法，同时变量选择和秩减少，发现在广泛的学科和领域，如机器学习，信号处理和生物统计学等应用。透过统计学、数学、工程学及计算机科学的交叉思想，整合研究与教育，透过跨学科训练，帮助学生发展批判性思维，并协助学生成为终身学习者。研究者利用这个项目中丰富的主题来激发公众和所有年龄段学生的学习和发现兴趣。教育计划包括课程开发、学生辅导、外联和招收代表性不足的学生。

The data explosion in all fields of science creates an urgent need for methodologies for analyzing high dimensional multivariate data. The project deepens and broadens existing sparsity and low rank statistical theories and methods by making the following major scientific achievements: (a) an innovative selectable reduced rank methodology through simultaneous variable selection and projection, with guaranteed lower error rate than existing variable selection and rank reduction rates in theory, which paves the way to new frontiers in high dimensional statistics and information theory; (b) fast but simple-to-implement algorithms that can deal with all popular penalty functions (possibly nonconvex) in computation with guaranteed global convergence and local optimality, to ensure the practicality of the proposed approaches in big data applications; (c) a generic extension to non-Gaussian models capable of taking into account the correlation between multivariate responses, with a universal algorithm design based on manifold optimization; (d) a unified robustification scheme that can both identify and accommodate gross outliers occurring frequently in real data, to overcome the non-robustness of many conventional multivariate tools; (e) general-purpose model selection methods serving variable selection and/or rank reduction and achieving the finite-sample optimal prediction error rate with theoretical guarantee. The need to recover low-dimensional signals from high dimensional multivariate noisy data permeates all fields of science and engineering. Hence a project of this nature, designed to develop transformative theory and methods for simultaneous variable selection and rank reduction, finds applications in a wide range of disciplines and areas such as machine learning, signal processing, and biostatistics, among others. By cross-fertilizing ideas from statistics, mathematics, engineering, and computer science, the integrated research and education help students develop critical thinking through cross-disciplinary training, and assist students in becoming life-long learners. The investigator uses the rich topics in this project to inspire the learning and discovery interest of the public and students of all ages. The educational plan consists of course development, student mentoring, outreach, and recruiting underrepresented students.