Sparsity Based Regularization
Sparsity Based Regularization
复制标题
基于稀疏性的正则化
DOI：
--
复制
发表时间：
2008
期刊：
影响因子：
--
通讯作者：
L. Rosasco
中科院分区：
文献类型：
--
作者：
L. Rosasco
在之前的讲座中，我们看到了如何使用正则化来恢复经验风险最小化（ERM）问题的适定性。我们还推导了使用正则化在解空间上施加平滑假设（如Tikhonov正则化的情况）或通过将解空间限制为低维流形（流形正则化）引入额外结构的算法。在这个讲座中，我们将研究正则化的使用，以实现另一个目标，即稀疏性。在过去的十年中，人们对稀疏性这个一般领域的兴趣越来越大。这种兴趣不仅来自机器学习社区，也来自其他科学领域。例如，在信号处理中，稀疏性主要在压缩感知[CRT06， Dono06]和所谓的基追踪[CDS96]的背景下进行研究。在统计学文献中，基础追求被称为套索[Tibs96]。稀疏编码[OlFi97]和独立分量分析[HyOj00]也存在强连接。在这些笔记中，我们从正则化的角度来讨论稀疏性，并且只参考这些连接，因为它们是从这个框架中产生的。最初，我们鼓励使用稀疏性并强调变量选择问题。然后，我们提出了基于稀疏性的正则化问题的公式，开发了易于处理的近似，并使用稀疏性的几何解释来证明它们。最后，在讨论了这些近似的一些性质之后，我们描述了一种求解基于稀疏性的正则化问题的算法。
In previous lectures, we saw how regularization can be used to restore the well-posedness of the empirical risk minimization (ERM) problem. We also derived algorithms that use regularization to impose smoothness assumptions on the solution space (as in the case of Tikhonov regularization) or introduce additional structure by confining the solution space to low dimensional manifolds (manifold regularization). In this lecture, we will examine the use of regularization for the achievement of an alternative objective, namely sparsity. During the last ten years, there has been an increased interest in the general field of sparsity. Such interest comes not only from the Machine Learning community, but also from other scientific areas. For example, in Signal Processing sparsity is examined mainly in the context of compressive sensing [CRT06, Dono06] and the so called basis pursuit [CDS96]. In the Statistics literature, basis pursuit is known as the lasso [Tibs96]. Strong connections also exist with sparse coding [OlFi97] and independent component analysis [HyOj00]. In these notes, we discuss sparsity from a regularization point of view, and only refer to these connections as they arise from within this framework. Initially, we motivate the use of sparsity and emphasize on the problem of variable selection. Then, we present the formulation of the sparsity based regularization problem, develop tractable approximations to it, and justify them using a geometric interpretation of sparsity. Finally, after discussion of some of the properties of these approximations, we describe an algorithm for the solution of the sparsity based regularization problem.