An Integrated Linguistic-Computational Approach to Automatic Recognition of American Sign Language
An Integrated Linguistic-Computational Approach to Automatic Recognition of American Sign Language
批准号:
9905848
负责人:
Ronnie Wilbur
金额:
$50.4万
依托单位国家:
美国
项目类别:
Continuing Grant
财政年份:
1999
资助国家:
美国
项目状态:
已结题
起止时间:
1999-11-01 至 2004-10-31
中文摘要
该项目为美国手语(ASL)的自动识别并最终实现自动翻译提供了一种新的方法。该方法利用了这样一个事实,即符号是由部件(手形、位置、方向、运动)组成的,就像单词由辅音和元音组成一样。手势识别不是试图将输入与存储的一组词汇手势进行匹配,而是将手势识别作为几个独立但相互关联的部件识别过程的最终结果,其中当前项目的重点是手形识别。这种设计消除了与存储和检索查找词典中的手势3-D运动图像相关的巨大存储和处理时间困难。手部运动将由隐马尔可夫模型的参数来表示;此外,该方法旨在从一组可能的ASL手形中识别输入手形。当与识别位置、方向和运动的类似程序相结合时,复合识别的组件集应该在词典查找中产生单个词汇符号。该项目独特地整合了几个基础研究领域:关于美国手语中符号结构的语言学研究,关于人类对美国手语认知的心理语言学研究,以及来自统计模式识别和计算机视觉的先进技术,以分析来自手语立体图像的输入。该程序的设计使得有关ASL手形的语言信息可以用来帮助计算机确定它“看到”的是哪种手形。长期目标是将识别过程整合为ASL-英语机器翻译设备的用户界面。这样的设备将支持实际环境中签名者和发言者之间的互动,包括工作场所和教室。
英文摘要
This project offers a novel approach to the problem of automatic recognition, and eventually automatic translation, of American Sign Language (ASL). The approach takes advantage of the fact that signs are composed of components (handshape, location, orientation, movement), in much the same way that words are composed of consonants and vowels. Instead of attempting sign recognition as a process of matching the input against a stored set of lexical signs, sign recognition will be treated as the end result of several separate but interrelated component recognition procedures, of which the current project focuses on handshape recognition.This design eliminates the enormous storage and processing-time difficulties associated with storing and retrieving 3-D moving images of signs in a look-up lexicon. Hand motion will be represented by the parameters of Hidden Markov Models; additionally, the approach aims to identify an input handshape from among the set of possible ASL handshapes. When coupled with similar procedures that identify locations, orientation, and movement, the composite identified set of components should yield a single lexical sign in a dictionary look-up.The project uniquely integrates several areas of basic research: linguistic research on the structure of signs in ASL, psycholinguistic research on human perception of ASL, and advanced techniques from statistical pattern recognition and computer vision to analyze input from stereo images of signers. The procedure is designed so that linguistic information about ASL handshapes can be used to assist the computer in deciding which handshape it "sees". The long-term goal is to integrate the recognition procedure as the user interface of an ASL-English machine translation device. Such a device would support interactions between signers and speakers in practical settings, including the workplace and classrooms.
期刊论文(0)
专著(0)
科研奖励(0)
会议论文
Conference: Theoretical Issues in Sign Language Research 10
-
批准号:0926532
-
项目类别:Standard Grant
-
资助金额:$2.0万
-
财政年份:2009
-
负责人:Ronnie Wilbur
-
依托单位:
Software for Math Education for the Deaf
-
批准号:0622900
-
项目类别:Standard Grant
-
资助金额:$30.0万
-
财政年份:2006
-
负责人:Ronnie Wilbur
-
依托单位:
An integrated linguistic-computational approach to automatic recognition of American Sign Language
-
批准号:0414953
-
项目类别:Continuing Grant
-
资助金额:$23.8万
-
财政年份:2004
-
负责人:Ronnie Wilbur
-
依托单位:
A Basic Grammar of Croatian Sign Language
-
批准号:0345314
-
项目类别:Continuing Grant
-
资助金额:$39.99万
-
财政年份:2004
-
负责人:Ronnie Wilbur
-
依托单位:
Syllable Structure and Phonological Rules in American Sign Language
-
批准号:8317572
-
项目类别:Continuing Grant
-
资助金额:$17.57万
-
财政年份:1984
-
负责人:Ronnie Wilbur
-
依托单位:
海外基金