RI: Small: The Shape of Visual Motion

这个项目研究描述视频中运动的方法。世界上所有可见的点都通过它们的身份进行标记，它们在图像平面上的投影轨迹通过空间和时间进行跟踪。这种计算在空间和时间上都是全局执行的，并且运动间断在输出中被明确地描绘出来。以前的技术主要是自下而上地估计运动，一次从两帧开始，与此不同的是，来自摄像机的数据盒被分割成管状区域，其形状捕捉有关场景中可见对象的运动和变形的信息。新的方法包括通过低阶矩阵数据填充将所有视觉运动投影到点轨迹的稀疏基础上；在保留边界的函数空间中使用L1正则化；从变分和二次规划中推广稳健估计方法，用于在多帧情况下高效地计算管子和遮挡；以及几种特定领域的技术，用于初始化接近全局解的一般但局部优化方法。由此产生的描述符能够实现视频检索、心率异常的医疗诊断、体育运动中的表现评估、手语识别、交通监控、监控等等。该项目还构成了一个新的计算机视觉实验方法课程的基础，该课程的材料可以在网上获得。

This project studies methods for describing motion in video. All visible points in the world are tagged by their identity, and trajectories of their projections on the image plane are tracked through space and time. This computation is performed globally, both in space and time, and motion discontinuities are explicitly delineated in the output. In contrast with previous techniques, which estimate motion primarily from the bottom up, starting with two frames at a time, the box of data from a video camera is carved up into tube-like regions whose shapes capture information about the motion and deformation of the objects visible in the scene. Novel methods include the projection of all visual motion onto a sparse basis of point trajectories through low-rank matrix data imputation; the use of L1 regularization in a function space that preserves boundaries; the generalization of robust estimation methods from variational calculus and quadratic programming for the efficient computation of tubes and occlusions in the multi-frame case; and several domain-specific techniques for initializing general but local optimization methods close to the global solution. The resulting descriptors enable video retrieval, medical diagnosis of heart rhythm anomalies, assessment of performance in sports, sign language recognition, traffic monitoring, surveillance, and more. The project also forms the basis for a new class on experimental methods for computer vision, the materials of which are made available online.