Real-time gesture recognition system and application
Real-time gesture recognition system and application
复制标题
实时手势识别系统及应用
DOI：
10.1016/s0262-8856(02)00113-0
复制
发表时间：
2002
期刊：
影响因子：
--
通讯作者：
S. Ranganath
中科院分区：
文献类型：
--
作者：
Chan Wah Ng;S. Ranganath
在本文中，我们考虑了一个基于视觉的系统，该系统可以实时解释用户的手势，以操纵图形用户界面中的窗口和对象。手分割程序首先从获取的图像序列的每一帧中提取二值手斑点。傅里叶描述子用于表示手团的形状，并被输入到径向基函数（RBF）网络中进行姿态分类。来自RBF网络输出的姿态似然向量与运动信息一起被用作手势识别器的输入。研究了隐马尔可夫模型（HMM）和递归神经网络（RNN）的手势识别性能。测试结果表明，连续HMM的手势识别率为90.2%，表现最佳。结合连续hmm和rnn的实验表明，两种分类器的线性组合将分类结果提高到91.9%。该手势识别系统部署在一个原型用户界面应用程序中，测试用户发现手势直观，应用程序易于使用。实时处理速率高达每秒22帧。
In this paper, we consider a vision-based system that can interpret a user's gestures in real time to manipulate windows and objects within a graphical user interface. A hand segmentation procedure first extracts binary hand blob(s) from each frame of the acquired image sequence. Fourier descriptors are used to represent the shape of the hand blobs, and are input to radial-basis function (RBF) network(s) for pose classification. The pose likelihood vector from the RBF network output is used as input to the gesture recognizer, along with motion information. Gesture recognition performances using hidden Markov models (HMM) and recurrent neural networks (RNN) were investigated. Test results showed that the continuous HMM yielded the best performance with gesture recognition rates of 90.2%. Experiments with combining the continuous HMMs and RNNs revealed that a linear combination of the two classifiers improved the classification results to 91.9%. The gesture recognition system was deployed in a prototype user interface application, and users who tested it found the gestures intuitive and the application easy to use. Real time processing rates of up to 22 frames per second were obtained.