RI: Small: The Shape of Visual Motion

本项目研究在视频中描述运动的方法。世界上所有可见的点都被标记为它们的身份，它们在图像平面上的投影轨迹通过空间和时间被跟踪。 该计算在空间和时间上全局执行，并且在输出中明确描绘运动不连续性。与以前的技术，估计运动主要是从下到上，从两个帧的时间开始，从视频摄像机的数据框被分割成管状区域，其形状捕捉有关的运动和变形的对象在场景中可见的信息。新的方法包括通过低秩矩阵数据填充将所有视觉运动投影到点轨迹的稀疏基上;在保留边界的函数空间中使用L1正则化;从变分法和二次规划中推广鲁棒估计方法，用于在多帧情况下有效计算管道和遮挡;以及几种用于初始化接近全局解的一般但局部优化方法的特定于域的技术。由此产生的描述符可以实现视频检索、心律异常的医学诊断、运动表现评估、手语识别、交通监控、监视等。该项目还构成了一个新的计算机视觉实验方法课程的基础，其材料可在网上获得。

This project studies methods for describing motion in video. All visible points in the world are tagged by their identity, and trajectories of their projections on the image plane are tracked through space and time. This computation is performed globally, both in space and time, and motion discontinuities are explicitly delineated in the output. In contrast with previous techniques, which estimate motion primarily from the bottom up, starting with two frames at a time, the box of data from a video camera is carved up into tube-like regions whose shapes capture information about the motion and deformation of the objects visible in the scene. Novel methods include the projection of all visual motion onto a sparse basis of point trajectories through low-rank matrix data imputation; the use of L1 regularization in a function space that preserves boundaries; the generalization of robust estimation methods from variational calculus and quadratic programming for the efficient computation of tubes and occlusions in the multi-frame case; and several domain-specific techniques for initializing general but local optimization methods close to the global solution. The resulting descriptors enable video retrieval, medical diagnosis of heart rhythm anomalies, assessment of performance in sports, sign language recognition, traffic monitoring, surveillance, and more. The project also forms the basis for a new class on experimental methods for computer vision, the materials of which are made available online.