Distributed Recursive Robust Estimation: Theory, Algorithms and Applications in Single and Multi-Camera Computer Vision

许多计算机视觉问题需要递归鲁棒估计。一些示例包括涉及由静态相机网络监视的大型室内或室外区域的监视应用;自然背景场景的恢复，例如具有移动的树叶和树枝的森林场景，以及动画电影中纹理合成应用的子空间估计;以及多站点视频会议。监视问题需要跟踪移动对象;如果背景是静态的，这可能是一个简单的问题。然而，考虑在雨天或雾天进行户外场景监测。随着雾的移动及其密度的变化，会导致背景复杂且不断变化，跟踪算法需要对这种类型的“大”背景噪声具有鲁棒性。如果没有遮挡（在这种情况下，背景序列是直接可用的），动画的纹理合成是一个研究得很好的问题，但在存在严重（大尺寸和持久性）遮挡的情况下变得困难，例如，移动和偶尔静止的动物遮挡背景场景。在这个项目中，我们表明，在所有上述问题中最具挑战性的步骤可以是分布式递归鲁棒主成分分析（PCA）问题，这是强大的离群值，或作为一个分布式递归鲁棒稀疏恢复问题，这是强大的，但结构化的噪声（噪声是非稀疏的，位于低维子空间）。这个项目的主要目标是开发分布式算法来解决多摄像机设置的这些问题。这些算法将在多站点视频组合应用程序（多站点视频会议所需）的背景下开发。 该项目正在开发第一套在线分布式解决方案，用于将矩阵分解为稀疏矩阵和低秩矩阵的总和。鲁棒PCA和鲁棒稀疏恢复是这个更一般问题的特殊情况。与现有的批处理方法相比，我们的在线解决方案将显著更快，内存效率更高。此外，与大多数批处理方法不同，即使对于缓慢移动或偶尔静态的前景对象，这些方法也可以证明有效（这些导致稀疏矩阵也变得秩不足，因此批处理方法在这种情况下不起作用）。这一优势是因为我们的方法利用了准确的初始子空间知识和缓慢的子空间变化（两者通常在真实的视频中是实际有效的假设）。我们在计算机视觉文献中工作的关键新奇在于，它对缓慢变化的背景或频繁和持续的遮挡（取决于前景或背景是否是感兴趣的层）具有鲁棒性。

Many computer vision problems require recursive robust estimation. Some examples include surveillance applications involving a large indoor or outdoor area monitored by a network of static cameras; natural background scenes' recovery, e.g. forest scenes with moving leaves and branches, and their subspace estimation for texture synthesis applications in animated movies; and multi-site video conferencing. The surveillance problem requires tracking moving objects; this can be an easy problem if the background is static. However consider outdoor scene monitoring on a rainy or very foggy day. As the fog moves and its density changes, it results in complex and changing backgrounds and the tracking algorithms need to be robust to this type of 'large' background noise. The texture synthesis for animation is a well-studied problem if there are no occlusions (in this case the background sequence is directly available) but becomes difficult in the presence of severe (large-sized and persistent) occlusions, e.g. moving and occasionally static animals occluding the background scenes. We show in this project that the most challenging step in all the above problems can either be posed as a distributed recursive robust principal components' analysis (PCA) problem, that is robust to outliers, or as a distributed recursive robust sparse recovery problem, that is robust to large but structured noise (noise that is non-sparse and lies in a low-dimensional subspace). The main goal of this project is to develop distributed algorithms to solve these problems for the multi-camera setting. The algorithms will be developed in the context of a multi-site video combining application (needed for multi-site video conferencing). This project is developing the first set of online distributed solutions for the decomposition of a matrix into a sum of a sparse and a low-rank matrix. Robust PCA and robust sparse recovery are special cases of this more general problem. Our online solutions will be significantly faster and memory-efficient compared to existing batch methods. Moreover, unlike most batch methods, these will provably work even when for slow moving or occasionally static foreground objects (these result in the sparse matrix also becoming rank deficient and hence batch methods do not work in this case). This advantage comes because our methods exploit accurate initial subspace knowledge and slow subspace change (both are usually practically valid assumptions in real videos). The key novelty of our work within the computer vision literature is that it is robust to slow changing backgrounds or to frequent and persistent occlusions (depending whether the foreground or the background is the layer of interest).