Distributed Recursive Robust Estimation: Theory, Algorithms and Applications in Single and Multi-Camera Computer Vision

许多计算机视觉问题需要递归鲁棒估计。一些例子包括涉及由静态摄像机网络监控的大型室内或室外区域的监视应用；自然背景场景的恢复，如树叶和树枝移动的森林场景及其子空间估计在动画电影纹理合成中的应用以及多站点视频会议。监控问题需要跟踪移动的物体；如果背景是静态的，这可能是一个容易的问题。然而，考虑在雨天或大雾天进行室外场景监控。随着雾的移动及其密度的变化，它会导致复杂和不断变化的背景，跟踪算法需要对这种“大”背景噪声具有鲁棒性。如果没有遮挡（在这种情况下背景序列是直接可用的），动画的纹理合成是一个很好的研究问题，但是在存在严重（大尺寸和持续）遮挡的情况下变得困难，例如移动和偶尔静止的动物遮挡背景场景。我们在这个项目中表明，上述所有问题中最具挑战性的步骤可以作为一个分布式递归鲁棒主成分分析（PCA）问题，它对异常值具有鲁棒性，或者作为一个分布式递归鲁棒稀疏恢复问题，它对大但结构化的噪声（非稀疏且位于低维子空间的噪声）具有鲁棒性。本项目的主要目标是开发分布式算法来解决多摄像机设置中的这些问题。这些算法将在多站点视频组合应用（多站点视频会议所需）的背景下开发。该项目正在开发第一组在线分布式解，用于将矩阵分解为稀疏矩阵和低秩矩阵的和。鲁棒PCA和鲁棒稀疏恢复是这个更普遍问题的特殊情况。与现有的批处理方法相比，我们的在线解决方案将显著更快，内存效率更高。此外，与大多数批处理方法不同，即使对于缓慢移动或偶尔静态的前景对象，这些方法也可以证明是有效的（这会导致稀疏矩阵也变得秩不足，因此批处理方法在这种情况下不起作用）。这种优势是因为我们的方法利用了精确的初始子空间知识和缓慢的子空间变化（这两者通常在实际视频中都是有效的假设）。我们在计算机视觉文献中工作的关键新颖之处在于，它对缓慢变化的背景或频繁和持续的遮挡（取决于前景或背景是感兴趣的层）具有鲁棒性。

Many computer vision problems require recursive robust estimation. Some examples include surveillance applications involving a large indoor or outdoor area monitored by a network of static cameras; natural background scenes' recovery, e.g. forest scenes with moving leaves and branches, and their subspace estimation for texture synthesis applications in animated movies; and multi-site video conferencing. The surveillance problem requires tracking moving objects; this can be an easy problem if the background is static. However consider outdoor scene monitoring on a rainy or very foggy day. As the fog moves and its density changes, it results in complex and changing backgrounds and the tracking algorithms need to be robust to this type of 'large' background noise. The texture synthesis for animation is a well-studied problem if there are no occlusions (in this case the background sequence is directly available) but becomes difficult in the presence of severe (large-sized and persistent) occlusions, e.g. moving and occasionally static animals occluding the background scenes. We show in this project that the most challenging step in all the above problems can either be posed as a distributed recursive robust principal components' analysis (PCA) problem, that is robust to outliers, or as a distributed recursive robust sparse recovery problem, that is robust to large but structured noise (noise that is non-sparse and lies in a low-dimensional subspace). The main goal of this project is to develop distributed algorithms to solve these problems for the multi-camera setting. The algorithms will be developed in the context of a multi-site video combining application (needed for multi-site video conferencing). This project is developing the first set of online distributed solutions for the decomposition of a matrix into a sum of a sparse and a low-rank matrix. Robust PCA and robust sparse recovery are special cases of this more general problem. Our online solutions will be significantly faster and memory-efficient compared to existing batch methods. Moreover, unlike most batch methods, these will provably work even when for slow moving or occasionally static foreground objects (these result in the sparse matrix also becoming rank deficient and hence batch methods do not work in this case). This advantage comes because our methods exploit accurate initial subspace knowledge and slow subspace change (both are usually practically valid assumptions in real videos). The key novelty of our work within the computer vision literature is that it is robust to slow changing backgrounds or to frequent and persistent occlusions (depending whether the foreground or the background is the layer of interest).