SGER: Generating Animations of American Sign Language Classifier Predicates

美国手语（ASL）是一种完全自然的语言，其语言结构与英语不同，是美国大约50万聋人的主要交流方式。此外，由于他们在儿童时期的关键语言习得阶段听不到英语口语，美国大多数聋人高中毕业生的英语阅读水平只有四年级。由于英语识字率很低，而且英语和美国手语有如此不同的语言结构，美国的许多聋哑人可以从一种技术中受益，这种技术可以将英语文本翻译成美国手语的动画，由计算机屏幕上的虚拟人物表演。但之前的英语到美国手语的机器翻译项目只取得了有限的进展。这些项目没有产生实际的ASL动画，而是产生了语言的有限子集，从而允许他们回避许多重要的语言和动画问题，特别是无处不在的ASL语言结构，称为“分类器谓词”，这是翻译许多英语输入句子所必需的。分类器谓词是一种美国手语现象，在这种现象中，签名者使用他或她身体周围的空间来定位代表正在讨论的实体或概念的不可见物体；签名者的手显示了这些物体在空间中的运动和位置。分类器谓词是最不像口语或书面语元素的ASL现象，因此它们很难被机器翻译软件翻译。在这项研究中，pi和他们的研究生将在美国手语语言学、机器翻译和人工智能、3D图形模拟和人类动画的基础上，设计和实现一个能够从英语文本中产生分类器谓词动画的原型软件系统。在此过程中，他们将解决英语到美国手语翻译中一些最具挑战性的问题，目标是产生一个软件设计，可以作为一个完整的英语到美国手语机器翻译系统的未来实现的健壮框架。原型实现将具有足够的视觉质量和语言广度，以便对聋人母语手语手语的设计和输出动画的质量进行试点评估。更广泛的影响：这项研究将导致英语到美国手语机器翻译软件的重大进步，这将最终允许开发新的应用程序，为成千上万的英语读写能力低的聋哑人提供更好的信息、媒体和服务。电脑、字幕电视或无线寻呼机等设备可以向聋哑人展示一个虚拟人物表演美国手语的动画，而不是显示英文文本。为聋儿开发新颖的教育阅读应用软件，提高他们的英语读写能力。该项目还将使研究生接触到与美国手语和动画相关的研究问题，并将支持加劳德特大学为这些学生开设的夏季美国手语语言培训项目。

American Sign Language (ASL) is a full natural language, with a linguistic structure distinct from English, used as the primary means of communication for approximately one half million deaf people in the United States. Furthermore, because they are unable to hear spoken English during the critical language acquisition years of childhood, the majority of deaf high school graduates in the U.S. have only a fourth grade English reading level. Because of this low English literacy rate and because English and ASL have such different linguistic structure, many deaf people in the United States could benefit from technology that translates English text into animations of ASL performed by a virtual human character on a computer screen. But previous English-to-ASL machine translation projects have made only limited progress. Instead of producing actual ASL animations, these projects have produced restricted subsets of the language, thus allowing them to side-step many important linguistic and animation issues, including in particular the ubiquitous ASL linguistic constructions called "classifier predicates" that are required in order to translate many English input sentences. Classifier predicates are an ASL phenomenon, in which the signer uses the space around his or her body to position invisible objects representing entities or concepts under discussion; the signer's hands show the movement and location of these objects in space. Classifier predicates are the ASL phenomenon that is most unlike elements of spoken or written languages, and they are therefore difficult to translate by machine translation software. In this research the PIs and their graduate students will build on prior research in ASL linguistics, machine translation and artificial intelligence, 3D graphics simulation and human animation, to design and implement a prototype software system capable of producing animations of classifier predicates from English text. In doing so, they will address some of the most challenging issues in English-to-ASL translation, with the goal of producing a software design that can serve as a robust framework for future implementation of a complete English-to-ASL machine translation system. The prototype implementation will have sufficient visual quality and linguistic breadth to enable a pilot evaluation of the design and the quality of the output animations by deaf native ASL signers.Broader Impacts: This research will lead to significant advances in the state of the art relating to English-to-ASL machine translation software, which will eventually allow development of new applications to provide improved access to information, media and services for the hundreds of thousands of deaf Americans who have low English literacy. Instead of displaying English text, devices like computers, closed-captioned televisions, or wireless pagers could show deaf users an animation of a virtual human character performing ASL. Novel educational reading applications software for deaf children to promote English literacy skills could also be developed. The project will also expose the graduate students involved to research issues relating to ASL and animation, and will support a summer ASL language training program at Gallaudet University for these students.