Intelligent Support System for Learning Sign Language with Video

1.一种手语单词关键特征的提取与建模方法(1)手语视频特征提取我们开发了一种方法，通过对附着在手语说话者身上的彩色指节、拇指节和腕带进行颜色跟踪来分析手部运动和手指形状。该方法利用特征空间法提取主成分流向量作为关键特征。其次，我们的方法通过使用傅里叶描述符抵消流矢量的谐波内容。此外，该方法还可以校正大范围镜头拍摄时产生的图像畸变。(2)基于自信息的特征建模与关键特征提取，分别用运动音素和手指形状音素表达符号特征。接下来，我们提出了一种基于符号特征数据库中每个特征的自信息提取关键特征的方法。本文根据动作错误的严重程度将手语学习者的动作错误分为两级，并提出了基于路径搜索的动作错误特征匹配方法。我们的方法采用扩展的标准运动特征，其中包括轻微变形的误差特征。由于采用了检测系统，我们的检测系统可以同时提取运动误差并确定运动误差的等级，实现误差的实时反馈。我们提出了一种基于特征重要性和人类记忆过程的关键特征阶段性学习的记忆策略。通过一个手语单词的学习实验，我们证明了该方法在长期记忆中是有效的。

1.A Method for Extraction and Modeling of Key Features in Sign Language Words(1)Feature Extraction from Sign Language VideoWe have developed a method for analyzing hand motions and finger shapes by color-tracking of colored fingerstalls, thumbstalls and wrist bands that are attached to a sign language speaker. The method extracts principal component flow vectors as key features using eigenspace method. Next, our method cancels harmonic contents of flow vectors by using fourier descriptor. In addition, the method can correct image distortion occurred by filming with wide-range lens.(2)Feature Modeling and Key Feature Extraction Based on Self InformationWe expressed sign features by motion phonemes and finger shape phonemes. Next, we have proposed a method that extracts key features based on self information of each feature within sign feature database.2.Feature Matching MethodWe classify motion errors of sign language learners into two levels in terms of the error seriousness, and propose an feature matching method for each level by using path-searching. Our method employs extended standard motion features that include error features deformed slightly. Owing to the employment, our detection system extracts motion errors and determines their level simultaneously for realizing real-time error feedback.3.Leaning Control MethodWe propose a memory strategy by phased learning of key features in terms of feature importance and human memory processes. By an experiment of learning sign words, we show that our method is effective in long-term memorizing of the words.