Avatar-Guided Estimation of Human Shape and Motion

我们提出了一个研究计划，以开发视觉算法，推断人类的形式和行动，从视频序列。 该方法的主要特点是使用一个动画的人形化身，以提供对身体形状和运动的强烈期望。 使用线投影将身体轮廓图像的时间序列折叠成可以使用鲁棒的信号处理技术进行分析的四个二维图案。 为了识别活动，将从人的视频生成的模式与从人形化身生成的时空原型进行匹配。 基于矩的时空模式粗对齐方法将模型和数据模式进行配准，从而可以比较它们来分类视点和活动。 由此产生的粗对齐预测2D身体拓扑结构和闭塞在每个视频帧中，这使得基于薄板样条的2D非刚性形状匹配方法能够识别和描绘每个图像中的身体部位。 化身数据还预测在哪些图像帧中可以最可靠地进行哪些身体尺寸测量，从而导致3D身体形状、姿势和运动的高效且准确的恢复。 我们最初计划专注于观察人类步态。 步态分析与一般的活动分析问题有许多相同的挑战，即高自由度的关节运动，从2D角度看身体部位的遮挡，以及不同个体的特殊表现。 同时，活动的周期性性质简化了模型和数据序列的时间对齐，并提高了对噪声数据的整体顺应性。 基于视觉的步态分析的成功将使从骨科的运动捕捉到智能房间和监控的人体生物特征计算等应用成为可能。

We propose a research program to develop vision algorithms that infer human form and action from video sequences. The main feature of the approach is the use of an animated humanoid avatar to provide strong expectations on body shape and motion. A temporal sequence of body silhouette images is collapsed using line projections into four two-dimensional patterns that can be analyzed using robust signal processing techniques. To identify activities, patterns generated from video of a person are matched against spatio-temporal prototypes generated from the humanoid avatar. A moment-based method for coarse temporal and spatial pattern alignment brings model and data patterns into registration, so that they can be compared to classify viewpoint and activity. The resulting coarse alignment predicts 2D body topology and occlusion in each video frame, which enables a 2D nonrigid shape matching method based on thin-plate splines to identify and delineate body parts in each image. The avatar data also predicts which body dimension measurements can be made most reliably in which image frames, leading to efficient and accurate recovery of 3D body shape, pose and motion. We initially plan to focus on observations of human gait. Gait analysis shares many of the same challenges as the general activity analysis problem, namely high degree of freedom articulated motion, occlusion of body parts from 2D viewpoints, and idiosyncratic performance by different individuals. At the same time, the periodic nature of the activity simplifies temporal alignment of model and data sequences, and improves overall resiliance to noisy data. Success in vision-based gait analysis would enable applications ranging from motion capture for orthopedics to computation of human biometrics for smart rooms and surveillance.