Statistical Approach to Real-time Estimation of Shape and Pose of Hand and Fingers in Image Sequence

点击翻译按钮获取中文摘要

We propose a new approach to hand tracking and 3D motion capture of hand and fingers. This new motion capture system does not require markers be attached to hands.A hand has many joints. To reduce computational complexity, we propose to reduce dimensionality applying Independent component analysis to the joint angle data collected by a data glove, and as a result the hand shape can be represented by only 5 parameters.Another problem with hand tracking is that when a hand undergoes free motion, self-occlusion occurs frequently. To solve this problem, we use multiple calibrated cameras. With multiple viewpoints, parts that are not visible to one camera can still be visible to other cameras.For each image, we try to estimate the 5 parameters representing hand shape and the 6 parameters for hand position and pose (3 for rotation and 3 for translation). These parameters are determined to maximize overlapping between the input image and the back-projected 3D hand model. The optimum is searched using particle filtering, which is robust to local optimum.The effectiveness of this approach is proved by testing the program to real image sequences with the hand rotating and deforming freely. Experimental results show that the system is robust against occlusion.However, particle filtering is time-consuming and the system does not work in real-time yet.