CAREER: Statistical Analysis of Nonconvex Optimization in Unsupervised Learning

无监督学习技术已经广泛应用于搜索、排名、推荐系统、社交网络、在线广告、在线交通、虚拟助手等现实世界的应用中。在非监督学习方法的许多实际应用中，基于非凸优化的估计方法因其对大数据的可扩展性而变得方便。这种可伸缩性在计算机视觉和自然语言处理等各种应用中至关重要。然而，由于可能存在不利的局部极小值，非凸优化在计算上是不稳定的，甚至是不可行的。因此，基于全球最低估计值的统计特性可能不会为从业者提供有意义的指导。本项目旨在研究一系列无监督学习问题中非凸优化方法的局部最小估计的统计性质。通过了解计算可行性和统计效率之间的权衡，所提出的研究项目对于确定可靠的非凸框架具有重要意义。将为统计、数学和计算机科学等不同领域的合作提供一个新的平台。该活动计划通过理论和计算培训以及实际数据分析，吸引女性和代表性不足的少数民族学生学习科学、技术、工程和数学(STEM)领域。由于非凸优化方法对缺失数据和各种参数具有较强的适应性，因此本文主要针对基于低阶分解的无监督学习进行统计分析，如矩阵补全、稳健主成分分析、成对排序和网络表示等。非凸低阶因式分解景观分析的最新进展表明，如果(I)基本事实满足强结构假设；(Ii)不允许模型失配；(Iii)样本量大，则不存在伪局部极小值。相比之下，拟议的项目侧重于在更一般的环境下进行景观分析：第一，将提出无模型框架来研究非凸优化的几何性质，而不需要对数据进行结构性假设或精确的模型匹配；第二，将分析模型失配对非凸目标函数的景观的影响；第三，将建立基于局部最小值的估计的统计效率及其与固有维度和样本大小的关系；第四，将利用通用参数化低阶分解的代数结构，以便为广泛类型的无监督学习问题建立统一的景观分析。此外，为了测试建议方法的经验行为，该活动计划在学习到排名、预测网络分析和推荐系统中确定适当的基准数据集，以便将建议方法的经验表现与文献中的基线方法进行比较。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Unsupervised learning techniques have been widely used in real-world applications such as searching, ranking, recommender systems, social networks, online advertisements, online transportation, virtual assistants, and so on. In many practical applications of unsupervised learning methodology, nonconvex optimization based estimation methods are convenient due to their scalability to large data. This scalability is crucial in various applications such as computer vision and natural language processing. However, nonconvex optimization is computationally unstable or even infeasible in general due to unfavorable local minima that may exist. In consequence, statistical properties for global minimum based estimates may not provide meaningful guidelines for practitioners. This project aims to study the statistical properties of local minimum based estimates for nonconvex optimization methods in a range of unsupervised learning problems. The proposed research projects are significant in identifying reliable nonconvex frameworks by understanding the trade-off between computational feasibility and statistical efficiency. A new platform will be provided for collaborations across different fields such as statistics, mathematics and computer science. The activity is planned to engage female and underrepresented minority students in the study in Science, Technology, Engineering and Mathematics (STEM) fields through both theoretical and computational training and hands-on data analysis. Since nonconvex optimization methods are known to be adaptive to missing data and various parameterizations, the proposed projects are focused on the statistical analysis for low-rank factorization based unsupervised learning, such as matrix completion, robust PCA, pairwise ranking and network representation. Recent developments in landscape analysis for nonconvex low-rank factorization reveal that there could be no spurious local minima if (i) the ground truth satisfies strong structural assumptions; (ii) model mismatching is not permitted; (iii) the sample size is large. In contrast, the proposed project is focused on conducting the landscape analysis in more general settings: First, model-free frameworks will be proposed to study the geometric properties of nonconvex optimization without requiring structural assumptions on the data or exact model matching; Second, the effects of model mismatching on the landscape of the nonconvex objective functions will be analyzed; Third, statistical efficiencies for local minima based estimates and their relationship with the intrinsic dimensions and sample sizes will be established; Fourth, algebraic structures of general parameterized low-rank factorization will be exploited in order to establish a unified landscape analysis for a broad class of unsupervised learning problems. Moreover, in order to test the empirical behavior of the proposed methods, the activity is planned to identify appropriate benchmark datasets in learning-to-rank, predictive network analysis and recommendation systems in order to compare the empirical performances of the proposed approaches with baseline methods in the literature.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.