An integrated linguistic-computational approach to automatic recognition of American Sign Language

自动识别美国手语的集成语言计算方法

• 批准号: 0414953
• 负责人: Ronnie Wilbur
• 金额: $ 23.8万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2004
• 资助国家: 美国
• 起止时间: 2004-08-01 至 2007-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0414953&HistoricalAwards=false
• 关键词: integrated; linguistic; computational; approach; automatic

Wilbur 0414953 An integrated linguistic-computational approach to automatic recognition of American Sign LanguageWith National Science Foundation support, linguist Dr. Ronnie B. Wilbur and electrical engineer Dr. Avinash C. Kak will conduct two years of integrated linguistic-computational research on the problem of automatic recognition of American Sign Lanugage (ASL). This novel approach takes advantage of the fact that signs are composed of components (handshape, place of articulation, orientation, movement, and possible nonmanuals, such as face/head position), in much the same way that words are composed of consonants and vowels. Instead of attempting sign recognition as a process of matching the input against a stored set of lexical signs, sign recognition is treated as the end result of several separate but interrelated component recognition procedures, of which the current project focuses on handshape recognition. The approach aims to identify an input handshape from among the set of possible ASL handshapes. When coupled with similar procedures that identify place of articulation, orientation, movement, and nonmanuals, the composite identified set of components should yield a single lexical sign in a dictionary look-up, much like the original dictionary created by Stokoe and colleagues in 1965. The project uniquely integrates several areas of basic research: linguistic research on the structure of signs in ASL, psycholinguistic research on human perception of ASL, and advanced techniques from statistical pattern recognition and computer vision to analyze input from stereo images of signers. The procedure is designed so that linguistic information about ASL handshapes can be used to assist the computer in deciding which handshape it "sees".Two scientific questions motivate this research. First, the project contributes to our understanding of the general phonetic and phonological organization of ASL when computers and humans are compared on the formational properties they use to perceive and categorize different handshapes. Second the project contributes to the development of algorithmic approaches to computer vision and pattern recognition of moving 3-D deformable objects, a theoretical issue in itself, by using as the objects of analysis the set of ASL signs that move though space and change handshapes. In addition to its scientific merit, this project contributes to the long-term development of an ASL-English machine translation device. Such a device would support interactions between signers and speakers in practical settings, including the workplace and classrooms.

美国手语自动识别的综合语言学-计算方法在美国国家科学基金会的支持下，语言学家Ronnie B. Wilbur博士和电气工程师Avinash C. Kak博士将对美国手语自动识别问题进行为期两年的综合语言学-计算研究。这种新颖的方法利用了这样一个事实，即符号是由组成部分组成的（手形、发音位置、方向、运动，以及可能的非手势，如脸/头的位置），就像单词由辅音和元音组成一样。符号识别不是将输入与存储的词汇符号进行匹配的过程，而是将符号识别视为几个独立但相互关联的组件识别过程的最终结果，其中当前项目的重点是手型识别。该方法旨在从一组可能的美国手语手势中识别输入手势。当与识别发音位置、方向、运动和非手册的类似过程相结合时，复合识别组件集应该在字典查找中产生单个词法符号，很像Stokoe及其同事在1965年创建的原始字典。该项目独特地整合了几个基础研究领域：美国手语符号结构的语言学研究，人类对美国手语感知的心理语言学研究，以及统计模式识别和计算机视觉等先进技术，以分析手语立体图像的输入。该程序的设计是为了使有关美国手语手势的语言信息可以用来帮助计算机决定它“看到”的手势。两个科学问题激发了这项研究。首先，该项目有助于我们了解美国手语的一般语音和语音组织，当计算机和人类比较他们用来感知和分类不同手形的形成属性时。其次，该项目有助于计算机视觉和移动3d可变形物体的模式识别的算法方法的发展，这本身就是一个理论问题，通过使用通过空间移动和改变手形的ASL手势集作为分析对象。除了具有科学价值外，该项目还有助于美国手语-英语机器翻译设备的长期发展。这种设备将支持在实际环境中，包括工作场所和教室中，签名者和说话者之间的互动。