Avatar-Guided Estimation of Human Shape and Motion

我们提出了一个研究计划，开发视觉算法，从视频序列推断人类的形式和行动。该方法的主要特点是使用动画人形化身来提供对身体形状和运动的强烈期望。身体轮廓图像的时间序列使用线投影折叠成四个二维模式，可以使用鲁棒信号处理技术进行分析。为了识别活动，从一个人的视频中生成的模式与从人形化身生成的时空原型相匹配。一种基于矩量的粗时空模式对齐方法，将模型和数据模式进行配准，从而对视点和活动进行分类比较。由此产生的粗对齐预测了每个视频帧中的二维身体拓扑和遮挡，这使得基于薄板样条的二维非刚性形状匹配方法能够识别和描绘每个图像中的身体部位。化身数据还可以预测在哪个图像帧中可以最可靠地进行哪些身体尺寸测量，从而有效准确地恢复3D身体形状，姿势和运动。我们最初计划关注对人类步态的观察。步态分析与一般的活动分析问题有许多相同的挑战，即高度自由的关节运动，从2D视点遮挡身体部位，以及不同个体的特殊表现。同时，活动的周期性简化了模型和数据序列的时间对齐，并提高了对噪声数据的整体弹性。基于视觉的步态分析的成功将使应用范围从骨科的动作捕捉到智能房间和监视的人体生物识别计算。

We propose a research program to develop vision algorithms that infer human form and action from video sequences. The main feature of the approach is the use of an animated humanoid avatar to provide strong expectations on body shape and motion. A temporal sequence of body silhouette images is collapsed using line projections into four two-dimensional patterns that can be analyzed using robust signal processing techniques. To identify activities, patterns generated from video of a person are matched against spatio-temporal prototypes generated from the humanoid avatar. A moment-based method for coarse temporal and spatial pattern alignment brings model and data patterns into registration, so that they can be compared to classify viewpoint and activity. The resulting coarse alignment predicts 2D body topology and occlusion in each video frame, which enables a 2D nonrigid shape matching method based on thin-plate splines to identify and delineate body parts in each image. The avatar data also predicts which body dimension measurements can be made most reliably in which image frames, leading to efficient and accurate recovery of 3D body shape, pose and motion. We initially plan to focus on observations of human gait. Gait analysis shares many of the same challenges as the general activity analysis problem, namely high degree of freedom articulated motion, occlusion of body parts from 2D viewpoints, and idiosyncratic performance by different individuals. At the same time, the periodic nature of the activity simplifies temporal alignment of model and data sequences, and improves overall resiliance to noisy data. Success in vision-based gait analysis would enable applications ranging from motion capture for orthopedics to computation of human biometrics for smart rooms and surveillance.