Real-time Shape and Posture Estimation of Articulated Objects Based on Multiple kinds of Uncertain Feature Information Obtained from Image Sequences

本文的研究成果总结如下：1.研究内容：我们开发了一种基于手部区域轮廓图像的视频速率手部姿态测量系统，该系统由具有大内存的高速pc机组成，并通过g位局域网系统相互连接。该系统可以使用单目摄像头实时估计近16万种可能的形状（包括高度自遮挡的手部形状）。我们开发了一种在线方法来改进姿态估计的铰接模型的初始形状。该方法采用多维椭球体描述，有效地描述了大量可能的姿态参数。该方法可以在单目图像序列的每个时间步进行姿态估计后，对给定的初始模型形状进行细化，但在目前的计算环境下，该方法仍然是一个计算量大的过程，无法进行实时处理。本文提出了一种利用多摄像头图像、颜色特征、图像运动流和观察指尖数量对日语手语单词进行识别的方法。尽管这些图像特征都有不确定性，但我们开发了一个系统，该系统使用隐马尔可夫模型学习集成了所有信息，并将其实现到手势界面中。提出了一种基于切换线性动力学结构的时空手势模型。基于该模型，我们开发了一种在线方法，从复杂的背景中分割出形状变化的人手区域，并从多个手势模型中进行识别。

The summarized results of this research are as follows:1. We developed a video-rate hand posture measurement system based on hand region contour image, which consists of high speed PCs with large memories and are connected with each other by G-bit LAN system. The system can estimate almost 160000 possible shapes (including highly self-occluded hand shapes) in real-time using monocular camera setting.2. We developed an online method to refine the initial shape of the articulated model for posture estimation. The method efficiently describes a huge set of the possible posture parameters employing a multi-dimensional ellisoid description. The method can refine the given initial model shape after the posture estimation in each timestep from a monocular image sequences, while it is still computationally heavy process and cannot be processed in real-time under the current computing environment.3. We proposed a recognition method of Japanese Sign Language words using multiple camera images, color feature, image motion flows and observed number of the finger tips. Although each of these image features has uncertainty, we developed the system which integrates whole of the information using Hidden Markov Model learning and implements it into a gestural interface.4. We proposed a novel time-space gestural model based on Switching Linear Dynamics architecture. Based on this model, we developed an on-line method to segment a shape-changing human hand region from a complicated background and identify it from several gesture models.