Sparse/robust estimation and Kalman smoothing with nonsmooth log-concave densities: modeling, computation, and theory
Sparse/robust estimation and Kalman smoothing with nonsmooth log-concave densities: modeling, computation, and theory
复制标题
DOI：
10.5555/2567709.2567747
复制
发表时间：
2013-01
期刊：
影响因子：
--
通讯作者：
A. Aravkin;J. Burke;G. Pillonetto
中科院分区：
文献类型：
--
作者：
A. Aravkin;J. Burke;G. Pillonetto
我们引入了一类新的二次支持（QS）函数，其中许多已经在各种应用中发挥了至关重要的作用，包括机器学习，鲁棒统计推断，稀疏性提升和逆问题，如卡尔曼平滑。众所周知的QS惩罚的例子包括l2，Huber，l1和Vapnik损失。我们建立在一个双重表示QS功能，用它来表征必要的条件，解释这些功能为负对数的真实概率密度。这种解释为使用已知和新的QS损失函数进行统计建模奠定了基础，并且能够从简单的标量构建块构建具有指定均值和方差的非平滑多变量分布。本文的主要贡献是为各种学习应用提供了一个灵活的统计建模框架，以及一个用于估计的有效数值方法工具箱。特别是，QS损失函数的一个广泛的子类，称为分段线性二次（PLQ）惩罚，具有双重表示，可用于设计内点（IP）方法。IP方法解决非光滑优化问题的工作直接与光滑方程组表征其最优性。我们提供了几个数值例子，沿着的代码，可以用来解决一般PLQ问题。IP方法的效率取决于特定应用程序的结构。我们考虑一类动态逆问题使用卡尔曼平滑。该类包括各种各样的应用，其目的是从噪声输出样本开始，用已知的过程和测量模型重建动态系统的状态。在经典的情况下，假设高斯误差的过程和测量模型，这样的问题。我们表明，扩展的框架允许使用任意PLQ密度，并且所提出的IP方法解决了广义卡尔曼平滑问题，同时保持时间序列大小的线性复杂性，就像在高斯情况下一样。这将Mayne-Fraser和Rauch-Tung-Striebel算法的计算效率扩展到更广泛的非光滑设置，并包括许多最近提出的鲁棒和稀疏平滑器作为特例。
We introduce a new class of quadratic support (QS) functions, many of which already play a crucial role in a variety of applications, including machine learning, robust statistical inference, sparsity promotion, and inverse problems such as Kalman smoothing. Well known examples of QS penalties include the l2, Huber, l1 and Vapnik losses. We build on a dual representation for QS functions, using it to characterize conditions necessary to interpret these functions as negative logs of true probability densities. This interpretation establishes the foundation for statistical modeling with both known and new QS loss functions, and enables construction of non-smooth multivariate distributions with specified means and variances from simple scalar building blocks. The main contribution of this paper is a flexible statistical modeling framework for a variety of learning applications, together with a toolbox of efficient numerical methods for estimation. In particular, a broad subclass of QS loss functions known as piecewise linear quadratic (PLQ) penalties has a dual representation that can be exploited to design interior point (IP) methods. IP methods solve nonsmooth optimization problems by working directly with smooth systems of equations characterizing their optimality. We provide several numerical examples, along with a code that can be used to solve general PLQ problems. The efficiency of the IP approach depends on the structure of particular applications. We consider the class of dynamic inverse problems using Kalman smoothing. This class comprises a wide variety of applications, where the aim is to reconstruct the state of a dynamical system with known process and measurement models starting from noisy output samples. In the classical case, Gaussian errors are assumed both in the process and measurement models for such problems. We show that the extended framework allows arbitrary PLQ densities to be used, and that the proposed IP approach solves the generalized Kalman smoothing problem while maintaining the linear complexity in the size of the time series, just as in the Gaussian case. This extends the computational efficiency of the Mayne-Fraser and Rauch-Tung-Striebel algorithms to a much broader nonsmooth setting, and includes many recently proposed robust and sparse smoothers as special cases.