Slow Kill for Big Data Learning

大数据应用通常涉及大量样本和特征，并且经常受到异常值的污染，这给变量选择和参数估计带来了挑战。在实践中，用规定的基数拟合稀疏模型是一种常见的要求，但它与解决高度非凸和离散的问题有关。在这种非凸优化中使用多个起点是很常见的，但在大数据中通常是计算禁止的；需要新的具有成本效益的技术来减轻起点要求并确保最佳的统计准确性。此外，如何调整任意给定的损失函数以防止总异常值并达到高崩溃点是现代数据分析的主要挑战。该项目将研究创新和有效的统计方法，并进行严格的理论分析来回答这些问题。在这个项目中，教育与研究紧密结合，包括课程开发、学生指导、外展和招募代表性不足的学生。该项目将提出一种用于大规模变量选择的新型慢杀技术，该技术由具有迭代变化阈值和同时l2正则化的可扩展优化算法驱动。渐进式分位数控制、增长学习率和慢杀死中的自适应l2 -收缩这三个主要元素有坚实的理论支持，并且相对于推进和前向路径算法，它在迭代过程中减少问题规模的能力使其对大数据具有吸引力。项目中统计和优化之间的相互作用将揭示在某些规则条件下的低错误率和快速收敛，而不需要追求全局最优解。此外，将引入一个抗离群值估计框架，以鲁棒化超出标准似然设置的给定方法。它与修剪方法密切相关，但包含了所有样本的显式离群参数，这反过来又方便了计算和理论。使用慢杀，数据重采样的次数将大大减少，并且所获得的抗估计器在低维和高维都可以享受最小最大速率最优性。总体而言，该研究将为鲁棒稀疏学习创建新一代高维工具，以适应大数据应用中的连贯设计和总体异常值，深化和拓宽现有的统计和优化方法和理论。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Big-data applications typically involve large numbers of samples and features and are often contaminated with outliers, posing challenges for variable selection and parameter estimation. Fitting a sparse model with a prescribed cardinality is a common request in practice, but it is associated with solving a highly nonconvex and discrete problem. Using multiple starting points in such nonconvex optimization is common, but is often computationally prohibitive on big data; new cost-effective techniques are needed to alleviate the starting point requirement and ensure the best statistical accuracy. Moreover, how to adjust an arbitrarily given loss function to guard against gross outliers and achieve a high break-down point poses a major challenge for modern-day data analysis. The project will study innovative and efficient statistical methods and perform rigorous theoretical analysis to answer these questions. In this project, education is tightly coupled with research, consisting of course development, student mentoring, outreach, and recruiting underrepresented students.The project will propose a novel slow-kill technique for large-scale variable selection, motivated by a scalable optimization algorithm with iteration-varying threshold and simultaneous L2-regularization. The three main elements of progressive quantile control, growing learning rate and adaptive L2-shrinkage in slow kill have solid theoretical support, and its ability to reduce the problem size during the iteration, as opposed to boosting and forward pathwise algorithms, makes it attractive for big data. The interplay between statistics and optimization in the project will reveal tight error rates and fast convergence under some regularity conditions, without the need to pursue a globally optimal solution. Furthermore, a framework of outlier-resistant estimation will be introduced to robustify a given method beyond the standard likelihood setup. It has a close connection to the method of trimming but includes explicit outlyingness parameters for all samples, which in turn facilitates computation and theory. With slow kill, the number of data resamplings will be substantially reduced, and the obtained resistant estimators can enjoy minimax rate optimality in both low and high dimensions. Overall, the proposed research will create a new-generation high dimensional tool for robust sparse learning that can accommodate coherent designs and gross outliers in big data applications, to deepen and broaden existing methods and theory in statistics and optimization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.