Research on Practical Sign Language Interpretation by integration of image processing of hands and other information

1.颜色用于手和面部区域的提取。由于这些区域的颜色可能会因人而异，因此我们开发了一种在颜色空间中确定肤色区域的方法。在该方法中，首先使用先验肤色区域提取近似区域。根据提取的手部和人脸区域的特征对该区域进行细化，并提出了一种避免手部快速移动时背景颜色被包含的方法.当手和面部重叠时，首先在皮肤区域中确定面部区域，并且在皮肤区域的其余部分中搜索手。由于人脸区域没有精确确定，因此利用人脸区域的确定性在皮肤区域进行hads模板匹配.我们开发了一种从一组真实的样本中合成手语样本的方法，以增加用于HMM学习的样本。实验证明，虽然识别率有所提高，但提高的幅度不大。原因之一是由于特征提取不当，合成样本的特征不合适。改进了特征提取方法.对于手语单词的识别，我们使用没有分支的HMM。由于同一个手语单词的特征因人而异，我们不得不准备多个模型。我们已经开发了一种方法，建立HMM与分支，以促进模型的多个手语功能。为了识别手语识别的人脸表情，我们开发了一种方法，确定的位置和形状的人脸部分，并检测有用的功能，人脸表情。在一个小规模的实验中，典型的表情，如广告的问题，喜悦，愤怒的识别方法。

1. Color is used for extraction of hands and face regions. Because the color of these regions may be varied depending on persons, we developd a method of determining the skin color region in a color space. In this method, approximate regions are first extracted using an apriori skin color region. This region is refined based on the properties of extracted hands and face regions.In addition, we develop a method to avoid inclusion of the background colors while hands are moving quickly.2. When hands and face overlap, the face region is first determined in the skin region, and the rest of the skin region is searched for hands. Because the face region is not determned precisely, the certainty of the face region is used for matching hads templates in the skin region.3. We developed a method of synthesizing sign language samples from a set of real samples to increase samples for HMM learning. Although impovement of recognition rate is proved by experiments, the amount of improvement is small. One reason is that the features of synthesized samples are not suitable because of improper feature extraction. We imporved our feature extraction method.4. For recognition of sign language words, we used HMM without branching. Because the features of the same sign language word varies depending on persons, we had to prepare multiple models. We have developed a method of building HMM with branching to facilitate models of multiple sign language features.5. In order to recognize face expressions for sign language recognition, we developed a method of determining the position and shape of face parts, and detecting useful features of face face expressions. In a small scale experiment, typical expressions such ad question, joy, and anger are identified by the method.