基于三维视觉及形状匹配的全自由度自然手势识别

基于视觉的手势识别是利用视频手势进行自然人机交互的关键。受限于2D计算机视觉技术的约束，目前基于视觉的手势识别系统在实际推广中还存在许多技术难点，主要包括复杂背景下的手势分割；手势模型的高自由度所带来的识别难度；视角与遮挡造成的识别准确度差；以及动态手势跟踪与识别中的计算复杂度等问题。深度摄像机的推广应用为手势识别带来了新的机遇和挑战。本课题基于深度摄像机所获取的3D视频手势，以全自由度关节模型驱动的三维动态可变形虚拟手网格建模为切入点，重点研究三维关节变形体的部分形状匹配技术，通过形状检索及模板匹配技术获得静态手形关节参数估计。以手形参数为基础，融合手势的位置、方向及空间运动轨迹，课题进一步研究基于虚拟手模型的动手势参数轨迹与参考模板之间的弹性匹配问题,并考虑人手校正，提高系统的识别精度，从而为基于3D摄像机的动静态手势识别系统提供关键技术和支撑框架。

Direct use of the hand as an input device is an attractive method for providing natural human-computer interact. To this aim, the research on vision-based hand gesture recognition is a key. However, limited by the tranditional 2D computer vision technology, current vision based hand gesture recognition systems suffer a series of challenges which have to be overcome for the widespread use of this technology. The main difficulties incldue uncontrolled environment, high-dimensional parameters, view dependence and self occlusions, processing speed in rapid hand motion tracking. This project is based on the color and depth images obtained by a depth camera. The key idea for hand gesture recognition starts from a virtual hand model, which is an deformable hand mesh driven by a full DOF skeletal model. We propose to use the template based posture estimation technique to fulfill the static posture recognition task. The 3D partial shape matching techniques are to be studied to obtain the set of full DOF joint parameters from the input partial hand surfaces. Based on the hand posture parameters, combining the location, orientation and kinematic parameters, we further study the problem of elastic matching between the template and the parametric trajectory for the input temporal posture. Automated hand caliberation problem will be tackled to improve the precision of a general purpose system. This research will provide the framework together with some key techniques for a three dimensional vision based hand posture and gesture recognition system.