权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Computational Methods for the Study of American Sign Language Nonmanuals Using Very Large Databases

使用大型数据库研究美国手语非手册的计算方法

基本信息

批准号：
9841303
负责人：
Aleix M Martinez
金额：
$ 31.78万
依托单位：
OHIO STATE UNIVERSITY
依托单位国家：
美国
项目类别：
财政年份：
2016
资助国家：
美国
起止时间：
2016-01-01 至 2023-12-31
项目状态：
已结题

来源：
https://reporter.nih.gov/project-details/9841303
关键词：
3-Dimensional Academic achievement Access to Information Address Agreement Algorithms American Sign Language Articulation Behavioral Child Code Communication Computational algorithm Computer Vision Systems Computer software Computing Methodologies Databases Detection Devices Emotions Excision Face Facial Expression Facial Muscles Goals Hand Head Hearing Human Image Individual Intervention Life Linguistics Logic Machine Learning Manuals Methods Movement Parents Pattern Recognition Production Research Research Personnel Science Semantics Series Shapes Sign Language Signal Transduction Specific qualifier value Speech Structure System Teacher Professional Development Technology Testing Time Visual Visual system structure base body position comparative computerized tools deaf deafness design experience experimental study face perception innovation instructor interest machine learning algorithm prevent public health relevance reconstruction showing emotion syntax tool

项目摘要

DESCRIPTION (provided by applicant): American Sign Language (ASL) grammar is specified by the manual sign (the hands) and by the nonmanual components, which include the face. Our general hypothesis is that nonmanual facial articulations perform significant semantic and syntactic functions by means of a more extensive set of facial expressions than that seen in other communicative systems (e.g., speech and emotion). This proposal will systematically study this hypothesis. Specifically, we will study the following three hypotheses needed to properly answer the general hypothesis stated above: First, we hypothesize (H1) that the facial muscles involved in the production of clause-level grammatical facial expressions in ASL and/or their intensity of activation are more extensive than those seen in speech and emotion. Second, we hypothesize (H2) that the temporal structure of these facial configurations are more extensive than those seen in speech and emotion. Finally, we hypothesize (H3) that eliminating these ASL nonmanual makers from the original videos, drastically reduces the chances of correctly identifying the clause type of the signed sentence. To test these three hypotheses, we define a highly innovative approach based on the design of computational tools for the analysis of nonmanuals in signing. In particular, we will examine the following three specific aims. In Aim 1, we will build a series of computer algorithms that allow us to automatically (i.e., without the need of any human intervention) detect the face, its facial features as well as the automatic detection of the movements of the facial muscles and their intensity of activation. These tools will be integrated into ELAN, a standard software used for linguistic analysis. These tools will then be used to test six specific hypotheses to successfully study H1. In Aim 2, we define computer vision and machine learning algorithms to identify the temporal structure of ASL facial configurations and examine how these compare to those seen in speech and emotion. We will study six specific hypotheses to successfully address H2. Alternative hypotheses are defined in both aims. Finally, in Aim 3 we define algorithms to automatically modify the original videos of facial expression in ASL to eliminate the identified nonmanual markers. Native users of ASL will complete behavioral experiments to examine H3 and test potential alternative hypotheses. Comparative analysis with non-signer controls will also be completed. These studies will thus further validate H1 and H2. We provide evidence of our ability to successfully complete the tasks in each of these aims. These aims address a critical need; at present, the study of nonmanuals must be carried out by hand. To be able to draw conclusive results, it is necessary to study thousands of videos. The proposed computational approach supposes at least a 50-fold reduction in time compared to methods done by hand.

描述（由申请人提供）：美国手语（ASL）语法由手动符号（手）和非手动组件（包括面部）指定。我们的一般假设是，非手动面部发音通过比其他交际系统中更广泛的面部表情集来执行重要的语义和句法功能（例如，言语和情感）。本文将系统地研究这一假说。具体来说，我们将研究以下三个假设，以正确回答上述一般假设：首先，我们假设（H1），在美国手语的小句水平的语法面部表情和/或其激活强度的生产涉及的面部肌肉比在言语和情感中看到的更广泛。其次，我们假设（H2），这些面部结构的时间结构比言语和情感中看到的更广泛。最后，我们假设（H3），从原始视频中删除这些ASL非手动标记，大大降低了正确识别签名句子的子句类型的机会。为了验证这三个假设，我们定义了一个高度创新的方法，基于设计的计算工具的分析非手册签署。我们将特别审查以下三个具体目标。在目标1中，我们将构建一系列计算机算法，使我们能够自动（即，不需要任何人为干预）检测面部、其面部特征以及面部肌肉的运动及其激活强度的自动检测。这些工具将被纳入用于语言分析的标准软件ELAN。然后，这些工具将用于测试六个特定的假设，以成功地研究H1。在目标2中，我们定义了计算机视觉和机器学习算法来识别ASL面部配置的时间结构，并研究这些与语音和情感中所见的时间结构的比较。我们将研究六个具体的假设，以成功地解决H2。在这两个目标中定义了备选假设。最后，在目标3中，我们定义了自动修改ASL中面部表情的原始视频以消除所识别的非手动标记的算法。ASL的本地用户将完成行为实验，以检查H3和测试潜在的替代假设。还将完成与非签名者控制的比较分析。因此，这些研究将进一步验证H1和H2。我们提供证据证明我们有能力成功完成这些目标中的每一项任务。这些目标解决了一个关键的需求;目前，非手册的研究必须手工进行。为了能够得出结论性的结果，有必要研究成千上万的视频。所提出的计算方法假设至少减少了50倍的时间相比，手工完成的方法。