An integrated linguistic-computational approach to automatic recognition of American Sign Language

威尔伯0414953 美国手语自动识别的综合语言学-计算方法在美国国家科学基金会的支持下，语言学家罗尼博士B。威尔伯和电气工程师Avinash C. Kak将对美国手语（ASL）的自动识别问题进行为期两年的综合语言学-计算研究。这种新颖的方法利用了这样一个事实，即符号是由组件（手形，发音的地方，方向，运动，以及可能的非手动，如面部/头部位置）组成的，就像单词由辅音和元音组成一样。而不是试图作为一个匹配的输入对一组存储的词汇符号的过程中，符号识别被视为几个独立的，但相互关联的组件识别程序，其中目前的项目侧重于手形识别的最终结果。该方法旨在从可能的ASL手形集合中识别输入手形。当与识别发音位置、方向、运动和非手动的类似程序相结合时，复合识别的组件集应该在字典查找中产生单个词汇符号，就像Stokoe及其同事在1965年创建的原始字典一样。该项目独特地整合了几个基础研究领域：对美国手语中符号结构的语言学研究，对美国手语人类感知的心理语言学研究，以及从统计模式识别和计算机视觉分析签名者立体图像输入的先进技术。这个过程的设计是为了让手语手形的语言信息可以用来帮助计算机决定它“看到”的手形。首先，该项目有助于我们理解的一般语音和语音组织的ASL时，计算机和人类进行比较，他们用来感知和分类不同的手形的形成属性。第二，该项目有助于计算机视觉和移动的3-D可变形物体的模式识别的算法方法的发展，这本身就是一个理论问题，通过使用作为分析对象的一组ASL标志，通过空间移动和改变手形。除了其科学价值外，该项目还有助于ASL-英语机器翻译设备的长期开发。 这种设备将支持签名者和发言者在实际环境中，包括工作场所和教室中的互动。

Wilbur 0414953 An integrated linguistic-computational approach to automatic recognition of American Sign LanguageWith National Science Foundation support, linguist Dr. Ronnie B. Wilbur and electrical engineer Dr. Avinash C. Kak will conduct two years of integrated linguistic-computational research on the problem of automatic recognition of American Sign Lanugage (ASL). This novel approach takes advantage of the fact that signs are composed of components (handshape, place of articulation, orientation, movement, and possible nonmanuals, such as face/head position), in much the same way that words are composed of consonants and vowels. Instead of attempting sign recognition as a process of matching the input against a stored set of lexical signs, sign recognition is treated as the end result of several separate but interrelated component recognition procedures, of which the current project focuses on handshape recognition. The approach aims to identify an input handshape from among the set of possible ASL handshapes. When coupled with similar procedures that identify place of articulation, orientation, movement, and nonmanuals, the composite identified set of components should yield a single lexical sign in a dictionary look-up, much like the original dictionary created by Stokoe and colleagues in 1965. The project uniquely integrates several areas of basic research: linguistic research on the structure of signs in ASL, psycholinguistic research on human perception of ASL, and advanced techniques from statistical pattern recognition and computer vision to analyze input from stereo images of signers. The procedure is designed so that linguistic information about ASL handshapes can be used to assist the computer in deciding which handshape it "sees".Two scientific questions motivate this research. First, the project contributes to our understanding of the general phonetic and phonological organization of ASL when computers and humans are compared on the formational properties they use to perceive and categorize different handshapes. Second the project contributes to the development of algorithmic approaches to computer vision and pattern recognition of moving 3-D deformable objects, a theoretical issue in itself, by using as the objects of analysis the set of ASL signs that move though space and change handshapes. In addition to its scientific merit, this project contributes to the long-term development of an ASL-English machine translation device. Such a device would support interactions between signers and speakers in practical settings, including the workplace and classrooms.