Constraint-Based Visual Hand Gesture Analysis

视觉手/手指跟踪和分析具有许多重要的潜在应用，包括人机交互、康复和手语识别。实时视觉手/手指跟踪的主要障碍是配置空间的巨大维度。例如，用手指关节角度来表示姿势，其自由度约为27个。在这个高维空间中跟踪在计算上是不可能的。然而，手指的运动是高度受限的。我们建议使用基于3D模型的方法，其中模型将包括姿势和运动约束。跟踪是通过匹配真实手的图像和3D模型预测的手的图像来完成的。关键是如何利用约束条件来缩短搜索时间。在本研究阶段结束时，我们期望有一个良好的约束表示和一个良好的实时手/手指跟踪算法，并演示了虚拟对象操作应用的实时跟踪。

Visual hand/finger tracking and analysis has many important potential applications, including human-computer interaction, rehabilitation, and the recognition of sign languages. The major obstacle to real-time visual hand/finger tracking is the huge dimensionality of the configuration space. E.g., using the finger joint angles to represent the posture, the number of degrees of freedom is about 27. Tracking in this high- dimensional space is computationally impossible. However, finger movement is highly constrained. We propose to use a 3D model-based approach, where the model will include both posture and motion constraints. Tracking is done by matching the image of the real hand and that of the predicted view of the hand from the 3D model. The crux is to use the constraints to reduce the search time. By the end of the proposed period, we expect to have a good constraint representation and a good real-time hand/finger tracking algorithm, and to have demonstrated real-time tracking for the application of virtual object manipulation.