权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Computational Methods for the Study of American Sign Language Nonmanuals Using Very Large Databases

使用大型数据库研究美国手语非手册的计算方法

基本信息

批准号：
9199411
负责人：
Aleix M Martinez
金额：
$ 31.94万
依托单位：
OHIO STATE UNIVERSITY
依托单位国家：
美国
项目类别：
财政年份：
2016
资助国家：
美国
起止时间：
2016-01-01 至 2020-12-01
项目状态：
已结题

来源：
https://reporter.nih.gov/project-details/9199411
关键词：
Academic achievement Access to Information Address Agreement Algorithms American Sign Language Articulation Behavioral Child Code Communication Computational algorithm Computer Vision Systems Computer software Computing Methodologies Controlled Study Databases Detection Devices Dimensions Emotions Excision Face Facial Expression Facial Muscles Goals Hand Head Hearing Hearing Impaired Persons Human Image Individual Intervention Life Linguistics Logic Machine Learning Manuals Methods Movement Parents Pattern Recognition Production Research Research Personnel Science Semantics Series Shapes Sign Language Signal Transduction Specific qualifier value Speech Structure System Teacher Professional Development Technology Testing Time Visual Visual system structure base body position comparative computerized tools deafness design experience experimental study face perception innovation instructor interest prevent public health relevance reconstruction showing emotion syntax tool

项目摘要

DESCRIPTION (provided by applicant): American Sign Language (ASL) grammar is specified by the manual sign (the hands) and by the nonmanual components, which include the face. Our general hypothesis is that nonmanual facial articulations perform significant semantic and syntactic functions by means of a more extensive set of facial expressions than that seen in other communicative systems (e.g., speech and emotion). This proposal will systematically study this hypothesis. Specifically, we will study the following three hypotheses needed to properly answer the general hypothesis stated above: First, we hypothesize (H1) that the facial muscles involved in the production of clause-level grammatical facial expressions in ASL and/or their intensity of activation are more extensive than those seen in speech and emotion. Second, we hypothesize (H2) that the temporal structure of these facial configurations are more extensive than those seen in speech and emotion. Finally, we hypothesize (H3) that eliminating these ASL nonmanual makers from the original videos, drastically reduces the chances of correctly identifying the clause type of the signed sentence. To test these three hypotheses, we define a highly innovative approach based on the design of computational tools for the analysis of nonmanuals in signing. In particular, we will examine the following three specific aims. In Aim 1, we will build a series of computer algorithms that allow us to automatically (i.e., without the need of any human intervention) detect the face, its facial features as well as the automatic detection of the movements of the facial muscles and their intensity of activation. These tools will be integrated into ELAN, a standard software used for linguistic analysis. These tools will then be used to test six specific hypotheses to successfully study H1. In Aim 2, we define computer vision and machine learning algorithms to identify the temporal structure of ASL facial configurations and examine how these compare to those seen in speech and emotion. We will study six specific hypotheses to successfully address H2. Alternative hypotheses are defined in both aims. Finally, in Aim 3 we define algorithms to automatically modify the original videos of facial expression in ASL to eliminate the identified nonmanual markers. Native users of ASL will complete behavioral experiments to examine H3 and test potential alternative hypotheses. Comparative analysis with non-signer controls will also be completed. These studies will thus further validate H1 and H2. We provide evidence of our ability to successfully complete the tasks in each of these aims. These aims address a critical need; at present, the study of nonmanuals must be carried out by hand. To be able to draw conclusive results, it is necessary to study thousands of videos. The proposed computational approach supposes at least a 50-fold reduction in time compared to methods done by hand.