An Integrated Linguistic-Computational Approach to Automatic Recognition of American Sign Language

该项目为美国手语（ASL）的自动识别和最终自动翻译问题提供了一种新颖的方法。这种方法利用了一个事实，即符号是由成分（手形、位置、方向、运动）组成的，就像单词是由辅音和元音组成的一样。符号识别不再是将输入与存储的词汇符号进行匹配的过程，而是将符号识别视为几个独立但相互关联的组件识别过程的最终结果，其中当前项目的重点是手型识别。这种设计消除了在查找词典中存储和检索3-D移动图像的巨大存储和处理时间困难。手部运动用隐马尔可夫模型的参数表示；此外，该方法旨在从一组可能的ASL手势中识别输入手势。当与标识位置、方向和移动的类似过程结合使用时，复合标识的组件集应该在字典查找中产生单个词法符号。该项目独特地整合了几个基础研究领域：美国手语符号结构的语言学研究，人类对美国手语感知的心理语言学研究，以及统计模式识别和计算机视觉等先进技术，以分析手语立体图像的输入。该程序的设计是为了使有关美国手语手势的语言信息可以用来帮助计算机决定它“看到”的手势。长期目标是将识别程序集成为一个ASL-English机器翻译设备的用户界面。这种设备将支持在实际环境中，包括工作场所和教室中，签名者和说话者之间的互动。

This project offers a novel approach to the problem of automatic recognition, and eventually automatic translation, of American Sign Language (ASL). The approach takes advantage of the fact that signs are composed of components (handshape, location, orientation, movement), in much the same way that words are composed of consonants and vowels. Instead of attempting sign recognition as a process of matching the input against a stored set of lexical signs, sign recognition will be treated as the end result of several separate but interrelated component recognition procedures, of which the current project focuses on handshape recognition.This design eliminates the enormous storage and processing-time difficulties associated with storing and retrieving 3-D moving images of signs in a look-up lexicon. Hand motion will be represented by the parameters of Hidden Markov Models; additionally, the approach aims to identify an input handshape from among the set of possible ASL handshapes. When coupled with similar procedures that identify locations, orientation, and movement, the composite identified set of components should yield a single lexical sign in a dictionary look-up.The project uniquely integrates several areas of basic research: linguistic research on the structure of signs in ASL, psycholinguistic research on human perception of ASL, and advanced techniques from statistical pattern recognition and computer vision to analyze input from stereo images of signers. The procedure is designed so that linguistic information about ASL handshapes can be used to assist the computer in deciding which handshape it "sees". The long-term goal is to integrate the recognition procedure as the user interface of an ASL-English machine translation device. Such a device would support interactions between signers and speakers in practical settings, including the workplace and classrooms.