Regularized Linear Regression: A Precise Analysis of the Estimation Error
Regularized Linear Regression: A Precise Analysis of the Estimation Error
复制标题
DOI：
--
复制
发表时间：
2015-06
期刊：
影响因子：
--
通讯作者：
Christos Thrampoulidis;Samet Oymak;B. Hassibi
中科院分区：
文献类型：
--
作者：
Christos Thrampoulidis;Samet Oymak;B. Hassibi
非平滑的正规凸优化程序已成为一种有力的工具，可以从（可能压缩）噪音线性测量中恢复结构化信号（稀疏，低级别等）。最小化模型的损失函数的优化方法，其对观测值的观测值增加了结构化诱导的调节术语。 grouplasso，最小二型偏差方法等。我们开发了一个相当通用的框架，用于确定此类方法的精确预测性能保证（例如，均值越高），用于高斯测量集合的情况。在许多实际应用中出现的其他凸度假设下的Min-Max定理。简化的辅助优化（AO）问题，我们可以从中紧密地推断出原始（PO）的特性，例如最佳成本，最佳解决方案的规范等。我们的理论适用于一般损失功能和监管，并提供有关如何如何提供指导方针在某些结构特性（例如稀疏性水平，等级等）时，可以最佳调整正规器的关键。
Non-smooth regularized convex optimization procedures have emerged as a powerful tool to recover structured signals (sparse, low-rank, etc.) from (possibly compressed) noisy linear measurements. We focus on the problem of linear regression and consider a general class of optimization methods that minimize a loss function measuring the misfit of the model to the observations with an added structured-inducing regularization term. Celebrated instances include the LASSO, GroupLASSO, Least-Absolute Deviations method, etc.. We develop a quite general framework for how to determine precise prediction performance guaranties (e.g. mean-square-error) of such methods for the case of Gaussian measurement ensemble. The machinery builds upon Gordon’s Gaussian min-max theorem under additional convexity assumptions that arise in many practical applications. This theorem associates with a primary optimization (PO) problem a simplified auxiliary optimization (AO) problem from which we can tightly infer properties of the original (PO), such as the optimal cost, the norm of the optimal solution, etc. Our theory applies to general loss functions and regularization and provides guidelines on how to optimally tune the regularizer coefficient when certain structural properties (such as sparsity level, rank, etc.) are known.