权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

EAGER: Collaborative Research: Data Visualizations for Linguistically Annotated, Publicly Shared, Video Corpora for American Sign Language (ASL)

EAGER：协作研究：美国手语 (ASL) 语言注释、公开共享视频语料库的数据可视化

基本信息

批准号：
1748016
负责人：
Carol Neidle
金额：
$ 1.8万
依托单位：
Trustees of Boston University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2017
资助国家：
美国
起止时间：
2017-08-01 至 2018-07-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1748016&HistoricalAwards=false
关键词：
EAGER Collaborative Research Data Visualizations

项目摘要

Linguistic research on ASL has been held back by the lack of precise tools for measurement, over large corpora, of the non-manual articulations (i.e., facial expressions and head gestures) that carry key grammatical information in sign languages. The same limitations have, until now, also held back computer science research on sign language recognition and generation. The PIs have created valuable resources, through prior NSF support, to serve the research, education, and sign language communities, including: computational techniques for analysis of American Sign Language (ASL) videos and the SignStream software for linguistic annotation of sign language data; large linguistically annotated and computationally analyzed corpora with videos from native signers; and an online Data Access Interface (DAI) that enables intuitive and flexible searching, browsing, and download, to provide easy access to these publicly shared corpora. They have also exploited these corpora for research on the linguistic structure of ASL and on computer-based sign language recognition from video. Recently, they have developed new versions of SignStream and the DAI with many new features that are now ready to be released publicly. Both represent major improvements over earlier versions of these applications and, combined with the public release of large new richly annotated and readily searchable data sets, constitute resources that will be of great value to researchers, educators, and students in linguistics and computer science, by opening up whole new avenues of research and enabling dramatic improvements in computer-based sign language recognition and generation. The resulting wide-ranging research advances will also contribute to future computer-based applications that will enhance communication for and with deaf individuals, as well as applications that will have educational benefits and overall improve the lives of those who are deaf and hard-of-hearing. The part-time effort to be funded for the two key software developers will also enable them to provide the limited technical support that is essential during the first year of the public release of SignStream 3 and DAI 2.The goal of this project is to further improve the existing applications by incorporating several powerful enhancements and additional functionalities to enable the shared tools and data to support new kinds of research in both linguistics (for analysis of linguistic properties of ASL and other signed languages) and computer science (for work in sign language recognition and generation). Specifically, the PIs will incorporate into their displays, within both the annotation software and the Web interface, graphical representations of computer-generated analyses of ASL videos, so that users will be able to visualize the distribution and characteristics of key aspects of facial expressions and head movements that carry critical linguistic information in sign languages (e.g., head nods and shakes, eyebrow height, and eye aperture). The most challenging aspect of sign language generation has been the production of natural-looking, appropriately timed, facial expressions and head movements. The sophisticated approach to tracking and 3D modeling of such expressions that has been developed recently by Metaxas et al. makes it possible to derive precise information about these facial expressions and head gestures for large sets of video files.

ASL 的语言学研究一直受到阻碍，因为缺乏精确的工具来测量大型语料库中携带手语关键语法信息的非手动发音（即面部表情和头部姿势）。到目前为止，同样的局限性也阻碍了手语识别和生成的计算机科学研究。通过之前 NSF 的支持，PI 创造了宝贵的资源，为研究、教育和手语社区提供服务，包括：用于分析美国手语 (ASL) 视频的计算技术和用于手语数据语言注释的 SignStream 软件；大型语言注释和计算分析语料库，包含来自母语手语者的视频；以及在线数据访问接口 (DAI)，可实现直观灵活的搜索、浏览和下载，从而轻松访问这些公开共享的语料库。他们还利用这些语料库来研究 ASL 的语言结构和基于计算机的视频手语识别。最近，他们开发了新版本的 SignStream 和 DAI，其中包含许多新功能，现已准备好公开发布。两者都代表了这些应用程序早期版本的重大改进，并与公开发布的大量新的、注释丰富且易于搜索的数据集相结合，构成了对语言学和计算机科学领域的研究人员、教育工作者和学生具有巨大价值的资源，开辟了全新的研究途径，并实现了基于计算机的手语识别和生成的显着改进。由此产生的广泛的研究进展也将有助于未来基于计算机的应用程序，这些应用程序将增强聋哑人之间的沟通，以及具有教育效益并全面改善聋哑人和听力障碍人士的生活的应用程序。为两个关键软件开发人员提供的兼职工作也将使他们能够提供有限的技术支持，这在 SignStream 3 和 DAI 2 公开发布的第一年中至关重要。该项目的目标是通过合并几个强大的增强功能和附加功能来进一步改进现有应用程序，使共享工具和数据能够支持语言学（用于分析 ASL 和其他手语的语言特性）和计算机科学（用于工作）方面的新型研究。手语识别和生成）。具体来说，PI 将在注释软件和网络界面中将计算机生成的 ASL 视频分析的图形表示纳入其显示器中，以便用户能够可视化面部表情和头部运动关键方面的分布和特征，这些关键方面携带着手语中的关键语言信息（例如点头和摇头、眉毛高度和眼睛孔径）。手语生成最具挑战性的方面是生成看起来自然、时机恰当的面部表情和头部动作。 Metaxas 等人最近开发了对此类表情进行跟踪和 3D 建模的复杂方法。可以为大量视频文件获取有关这些面部表情和头部姿势的精确信息。