An Integrated Linguistic-Computational Approach to Automatic Recognition of American Sign Language

美国手语自动识别的综合语言计算方法

• 批准号: 9905848
• 负责人: Ronnie Wilbur
• 金额: $ 50.4万
• 依托单位: Purdue Research Foundation
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 1999
• 资助国家: 美国
• 起止时间: 1999-11-01 至 2004-10-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=9905848&HistoricalAwards=false
• 关键词: Integrated; Linguistic; Computational; Approach; Automatic

This project offers a novel approach to the problem of automatic recognition, and eventually automatic translation, of American Sign Language (ASL). The approach takes advantage of the fact that signs are composed of components (handshape, location, orientation, movement), in much the same way that words are composed of consonants and vowels. Instead of attempting sign recognition as a process of matching the input against a stored set of lexical signs, sign recognition will be treated as the end result of several separate but interrelated component recognition procedures, of which the current project focuses on handshape recognition.This design eliminates the enormous storage and processing-time difficulties associated with storing and retrieving 3-D moving images of signs in a look-up lexicon. Hand motion will be represented by the parameters of Hidden Markov Models; additionally, the approach aims to identify an input handshape from among the set of possible ASL handshapes. When coupled with similar procedures that identify locations, orientation, and movement, the composite identified set of components should yield a single lexical sign in a dictionary look-up.The project uniquely integrates several areas of basic research: linguistic research on the structure of signs in ASL, psycholinguistic research on human perception of ASL, and advanced techniques from statistical pattern recognition and computer vision to analyze input from stereo images of signers. The procedure is designed so that linguistic information about ASL handshapes can be used to assist the computer in deciding which handshape it "sees". The long-term goal is to integrate the recognition procedure as the user interface of an ASL-English machine translation device. Such a device would support interactions between signers and speakers in practical settings, including the workplace and classrooms.

该项目提供了一种新的方法来自动识别的问题，并最终自动翻译，美国手语（ASL）。这种方法利用了这样一个事实，即符号是由成分（手形，位置，方向，运动）组成的，就像单词是由辅音和元音组成的一样。符号识别将被视为几个独立但相互关联的组件识别过程的最终结果，而不是将符号识别作为将输入与存储的一组词汇符号相匹配的过程，其中目前的项目集中在手形识别。这种设计消除了巨大的存储和处理时间的困难，存储和检索三维运动图像的迹象，在一个外观-上词典手部运动将表示的隐马尔可夫模型的参数，此外，该方法的目的是从一组可能的ASL手形中识别输入的手形。当与识别位置、方向和运动的类似程序相结合时，复合识别组件集应该在字典查找中产生单个词汇符号。该项目独特地整合了基础研究的几个领域：对美国手语中符号结构的语言学研究，对人类对美国手语感知的心理语言学研究，以及统计模式识别和计算机视觉的先进技术，以分析签名者立体图像的输入。该程序的设计，使手语手形的语言信息可以用来帮助计算机决定它“看到”的手形。长期目标是将识别过程集成为ASL-英语机器翻译设备的用户界面。这种设备将支持签名者和发言者在实际环境中，包括工作场所和教室中的互动。