Multiple sensors system for sign language recognition

手语包括通过将手和手臂的形状、方向和运动相结合来建立交流的象征性手势，以及流畅地表达说话人思想的面部表情。美国手语是美国和加拿大50多万聋人的主要交流手段。我的研究重点是通过开发一个可以解释手语的软件工具来与听障人士进行交流。有两种方法可以收集关于手势的信息，数据手套和摄像机。数据手套是一种配备了传感器的手套，可以感知手的动作，并将这些动作与计算机连接。从数据手套获得的数据简明扼要，几乎不需要后处理。然而，数据手套受到传感器数量的限制，它们不能捕捉一些签名者用来传达额外信息的面部表情。最近的一些研究表明，摄像机可以捕捉手的位置和面部表情。然而，这类摄像头容易出现典型的计算机视觉问题，如遮挡和对照明条件的敏感度。这些基于视觉的方法通常使用图像处理技术将手从图像背景中分割出来，并提取诸如指尖的特征以用于手势识别。建议的系统依赖于手套和单个摄像头的输入。它由四个模块组成。首先，应用实时手部跟踪与提取算法对运动的手部进行跟踪，提取手部区域。其次，采用傅里叶描述子或小波描述子来刻画空间特征。将进行运动分析以表征时间特征。将创建具有空间和时间特征的新特征向量。在第三个模块中，将手形、方向和运动的描述作为训练模块的输入，用于分割和识别。采集到的数据将被传输到第四模块进行分类，并确定与该标志对应的单词。这项研究将允许听力受损的人与其他人交流。

Sign language consists of symbolic gestures to establish communication by combining the shape, orientation and movement of hands and arms, as well as facial expressions to fluidly express a speaker's thoughts. American Sign Language is the primary means of communication for more than half million deaf people in the United States and Canada. My research is focused on communication with the hearing-impaired by developing a software tool that can interpret sign language. There are two methods of collecting information about hand gestures, data gloves and video cameras. Data glove is a glove equipped with sensors that sense the movements of the hand and interfaces those movements with a computer. The data obtained from data gloves are concise and requires little post-processing. However, the data gloves are limited by the number of sensors and they do not capture facial expressions which some signers use to communicate additional information. Some recent research has been shown that video cameras can capture hands positions as well as facial expressions. However, such cameras are prone to typical computer vision problems such as occlusion and sensitivity to lighting conditions. These vision-based methods typically use image processing techniques to segment the hand from the image background, and extract features such as the fingertips for use in gesture recognition. The proposed system depends on the inputs from the gloves and a single camera. It consists of four modules. First, the real time hand tracking and extraction algorithm is applied to trace the moving hand and extract the hand region. Secondly, Fourier or wavelet descriptors will be adopted to characterize the spatial features. A motion analysis will be conducted to characterize the temporal features. The new feature vector will be created with both spatial and temporal features. In the third module, the description of hand shape, orientation, and movements are used as inputs to the training module for the segmentation and recognition. The collected data will be transferred to the fourth module for classification and the word corresponding to the sign will be determined. This research will allow a hearing-impaired person to communicate with other people.