CAREER: Statistical Analysis of Nonconvex Optimization in Unsupervised Learning

无监督学习技术已经广泛应用于搜索、排名、推荐系统、社交网络、在线广告、在线交通、虚拟助手等实际应用中，在许多无监督学习方法的实际应用中，基于非凸优化的估计方法由于其对大数据的可扩展性而非常方便。这种可扩展性在计算机视觉和自然语言处理等各种应用中至关重要。然而，非凸优化是计算不稳定的，甚至是不可行的，一般由于不利的局部极小值，可能存在。因此，全球最低估计数的统计特性可能无法为从业人员提供有意义的指导。本计画旨在研究非凸最佳化方法在一系列无监督学习问题中，基于局部最小值估计的统计性质。建议的研究项目是重要的，在确定可靠的非凸框架，通过理解计算的可行性和统计效率之间的权衡。将为统计、数学和计算机科学等不同领域的合作提供一个新的平台。该活动计划通过理论和计算培训以及实际数据分析，让女性和代表性不足的少数民族学生参与科学、技术、工程和数学领域的研究。由于非凸优化方法被认为是自适应的缺失数据和各种参数化，所提出的项目集中在基于低秩分解的无监督学习的统计分析，如矩阵完成，鲁棒PCA，成对排序和网络表示。非凸低秩因子分解的景观分析的最新发展表明，如果（i）地面真值满足强结构假设;（ii）不允许模型失配;（iii）样本量很大，则不可能存在虚假的局部极小值。相比之下，拟议的项目侧重于在更一般的环境中进行景观分析：首先，将提出无模型框架来研究非凸优化的几何特性，而不需要对数据或精确的模型匹配进行结构假设;其次，将分析模型失配对非凸目标函数景观的影响;第三，统计效率的局部最小值为基础的估计和它们的内在尺寸和样本量的关系将建立;第四，一般参数化的低秩分解的代数结构将被利用，以建立一个统一的景观分析广泛的一类无监督学习问题。此外，为了测试所提出的方法的经验行为，该活动计划在学习排名中识别适当的基准数据集，预测网络分析和推荐系统，以比较所提出的方法与文献中的基线方法的经验性能。该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的知识价值和更广泛的影响审查标准。

Unsupervised learning techniques have been widely used in real-world applications such as searching, ranking, recommender systems, social networks, online advertisements, online transportation, virtual assistants, and so on. In many practical applications of unsupervised learning methodology, nonconvex optimization based estimation methods are convenient due to their scalability to large data. This scalability is crucial in various applications such as computer vision and natural language processing. However, nonconvex optimization is computationally unstable or even infeasible in general due to unfavorable local minima that may exist. In consequence, statistical properties for global minimum based estimates may not provide meaningful guidelines for practitioners. This project aims to study the statistical properties of local minimum based estimates for nonconvex optimization methods in a range of unsupervised learning problems. The proposed research projects are significant in identifying reliable nonconvex frameworks by understanding the trade-off between computational feasibility and statistical efficiency. A new platform will be provided for collaborations across different fields such as statistics, mathematics and computer science. The activity is planned to engage female and underrepresented minority students in the study in Science, Technology, Engineering and Mathematics (STEM) fields through both theoretical and computational training and hands-on data analysis. Since nonconvex optimization methods are known to be adaptive to missing data and various parameterizations, the proposed projects are focused on the statistical analysis for low-rank factorization based unsupervised learning, such as matrix completion, robust PCA, pairwise ranking and network representation. Recent developments in landscape analysis for nonconvex low-rank factorization reveal that there could be no spurious local minima if (i) the ground truth satisfies strong structural assumptions; (ii) model mismatching is not permitted; (iii) the sample size is large. In contrast, the proposed project is focused on conducting the landscape analysis in more general settings: First, model-free frameworks will be proposed to study the geometric properties of nonconvex optimization without requiring structural assumptions on the data or exact model matching; Second, the effects of model mismatching on the landscape of the nonconvex objective functions will be analyzed; Third, statistical efficiencies for local minima based estimates and their relationship with the intrinsic dimensions and sample sizes will be established; Fourth, algebraic structures of general parameterized low-rank factorization will be exploited in order to establish a unified landscape analysis for a broad class of unsupervised learning problems. Moreover, in order to test the empirical behavior of the proposed methods, the activity is planned to identify appropriate benchmark datasets in learning-to-rank, predictive network analysis and recommendation systems in order to compare the empirical performances of the proposed approaches with baseline methods in the literature.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.