RI: Medium: Collaborative Research: Models of Handshape Articulatory Phonology for Recognition and Analysis of American Sign Language

RI：媒介：协作研究：用于识别和分析美国手语的手形发音音系模型

• 批准号: 1409886
• 负责人: Diane Brentari
• 金额: $ 27.55万
• 依托单位: University of Chicago
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-06-01 至 2019-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1409886&HistoricalAwards=false
• 关键词: RI; Medium; Collaborative; Research; Models

Sign languages are the primary means of communication for millions of Deaf people in the world, including about 350,000-500,000 American Sign Language (ASL) users in the US. While the hearing population has benefited from advances in speech technologies such as speech recognition and spoken web search, much less progress has been made for sign language interfaces. Advances depend on improved technology for analyzing sign language from video. In addition, the linguistics of sign language is less well-understood than that of spoken language. This project addresses both of these needs, with an interdisciplinary approach that will contribute to research in linguistics, language processing, computer vision, and machine learning. Applications of the work include better access to ASL social media video archives, interactive recognition and search applications for Deaf individuals, and ASL-English interpretation assistance.This project focuses on handshape in ASL, in particular on one constrained but very practical component: fingerspelling, or the spelling out of a word as a sequence of handshapes and trajectories between them. Fingerspelling comprises up to 35% of ASL, depending on the context, and includes 72% of ASL handshapes, making it an excellent testing ground. The project addresses gaps in existing work by focusing on handshape in various conditions, including fast, highly coarticulated signing. The main project activities include development of (1) robust automatic detection and recognition of fingerspelled words using new handshape models, including segmental and "multi-segmental" graphical models of ASL phonological features; (2) techniques for generalizing across signers, styles, and recording conditions; (3) improved phonetics and phonology of handshape, in particular contributing to an articulatory phonology of sign; and (4) publicly released multi-speaker, multi-style fingerspelling data and associated semi-automatic annotation.

符号语言是世界上数百万聋人的主要交流手段，其中包括美国约35万-500,000美国手语（ASL）用户。 尽管听力人群受益于语音技术（例如语音识别和口语搜索）的进步，但对于手语界面而言，进步少得多。 进步取决于改进的技术来分析视频的手语。 此外，手语的语言学比口语的语言不太理解。 该项目通过跨学科的方法解决了这两个需求，该方法将有助于语言学，语言处理，计算机视觉和机器学习。 这项工作的应用包括更好地访问ASL社交媒体视频档案，互动识别和聋人的搜索应用以及ASL-英语解释援助。该项目侧重于ASL中的握手，尤其是一个受约束但非常实用的组成部分：手指的组成部分，或者将单词拼写为手声和触手可及的序列。 根据上下文，手指最多占ASL的35％，其中包括72％的ASL握手，使其成为一个极好的测试场。 该项目通过在各种条件下（包括快速，高度共同的签名）在各种条件下关注握手来解决现有工作中的差距。 主要项目活动包括（1）使用新的握手模型（包括分段和“多段”的ASL语音特征图形模型，开发了（1）强大的自动检测和识别手指单词的识别； （2）用于跨签名，样式和记录条件概括的技术； （3）改善了握手的语音和语音学，特别是有助于符号的发音学； （4）公开发布的多扬声器，多式手指销售数据和相关的半自动注释。