CIF:Small: Theory and Methods for Simultaneous Feature Auto-grouping and Dimension Reduction in Supervised Multivariate Learning

现代现实世界的应用迫切需要分析和解释具有低维结构的高维数据。在存在大量响应变量的情况下，很少的特征可能与整个响应集完全无关；这导致基于稀疏性的变量选择无效和不可解释的普通低阶建模。为了解决这些问题，该项目建议根据特征对响应变量的贡献将其分组在可能的低维子空间中，以建立更简明和可解释的模型。在多变量学习的背景下，寻找聚类的内在代价和高维对信号恢复的潜在不利影响尚未被完全理解。大数据时代的另一个关键挑战是开发具有严格收敛保证的高效优化算法。由于问题的非凸性，所得到的算法解可能不是全局最优的，这使得统计误差分析不是平凡的。在聚类的背景下，关联的模型选择问题是另一个未解决的问题，最明显的是当特征的数量和/或响应的数量超过样本大小时。为了回答这些问题，正在引入创新和变革性的统计方法，并对拟议的算法进行分析，以证明其效率。该项目涵盖了机器学习、基因组学和宏观计量经济学等广泛领域的潜在应用，并将有助于交叉培养统计学、运筹学、经济学和生物工程的想法。教育活动与研究紧密结合，包括课程开发、学生指导、外展和招收代表性不足的学生。该项目提出了一种新颖的聚类降维学习框架，该框架利用联合矩阵正则化来放松基于稀疏性的学习的严格假设，并与普通的低阶建模相比获得可解释性。一些普遍的信息论限制揭示了在不考虑使用哪种估计器的情况下搜索集群的内在成本，以及在多变量学习中积累大量响应变量的好处。同时执行子空间学习和聚类的高效优化算法正在被开发；所得到的定点估计器虽然不一定是全局最优的，但仍然享有超过标准似然设置的期望的统计精度。最后，提出了一种新的信息准则，在不假定样本容量无限大或信噪比较大的情况下，提出了一种新的联合聚类和秩值选择准则。这项研究正在创造统计学、信息论、非凸优化和模型选择的融合，并结合现实世界的应用。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Modern real-world applications have created an urgent need for analyzing and interpreting high-dimensional data with low-dimensional structures. In situations where a large number of response variables is present, very few features may be completely irrelevant to the entire set of responses; this leads to ineffective sparsity-based variable selection and to non-interpretable vanilla low-rank modeling. To address these issues, this project proposes grouping the features based on their contributions to the response variables, in a possibly low-dimensional subspace, in order to build a more parsimonious and interpretable model. In the context of multivariate learning, the intrinsic cost of searching for clusters and the potential adverse effect of high-dimensionality on signal recovery are not yet fully understood. Another critical challenge in the big-data era is to develop efficient optimization algorithms with rigorous convergence guarantees. The fact that the obtained algorithmic solutions may not be globally optimal, due to the non-convexity of the problem, makes the statistical error analysis nontrivial. The associated model-selection problem is another unsolved problem in the context of clustering, most notably when the number of features and/or the number of responses go beyond the sample size. To answer these questions, innovative and transformative statistical methods are being introduced, and the proposed algorithms are being analyzed to demonstrate their efficiency. The project covers potential applications in a wide range of areas such as machine learning, genomics, and macro-econometrics, and will help cross-fertilize ideas from statistics, operations research, economics, and bio-engineering. Education activities are tightly coupled with research, and include course development, student mentoring, outreach, and recruiting underrepresented students. The project proposes a novel clustered reduced-rank learning framework that utilizes joint matrix regularizations to relax the stringent assumption of sparsity-based learning and to gain interpretability as compared with vanilla low-rank modeling. Some universal information-theoretic limits are revealing the intrinsic cost of searching for clusters regardless of the estimator in use, as well as the benefit of accumulating a large number of response variables in multivariate learning. Efficient optimization algorithm that perform simultaneous subspace learning and clustering are being developed; the resulting fixed-point estimators, while not necessarily globally optimal, still enjoy the desired statistical accuracy beyond the standard likelihood setup. Finally, a new kind of information criterion for joint cluster and rank selection is being proposed, without assuming either infinite sample size or large signal-to-noise ratio. The research is creating a fusion between statistics, information theory, nonconvex optimization, and model selection, with real-world applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.