Collaborative Research: CI-ADDO-EN: Development of Publicly Available, Easily Searchable, Linguistically Analyzed, Video Corpora for Sign Language and Gesture Research

合作研究：CI-ADDO-EN：开发公开可用、易于搜索、语言分析的视频语料库，用于手语和手势研究

• 批准号: 1059235
• 负责人: Vassilis Athitsos
• 金额: $ 6.66万
• 依托单位: University of Texas at Arlington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-08-01 至 2015-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1059235&HistoricalAwards=false
• 关键词: Collaborative; Research; CI; ADDO; EN

The goal of this project is to create a linguistically annotated, publicly available, and easily searchable corpus of video from American Sign Language (ASL). This will constitute an important piece of infrastructure, enabling new kinds of research in both linguistics and vision-based recognition of ASL. In addition, a key goal is to make this corpus easily accessible to the broader ASL community, including users and learners of ASL. As a result of our long-term efforts, we have an extensive collection of linguistically annotated video data from native signers of ASL. However, the potential value of these corpora has been largely untapped, notwithstanding their extensive and productive use by our team and others. Existing limitations in our hardware and software infrastructure make it cumbersome to search and identify data of interest, and to share data among our institutions and with other researchers. In this project, we propose hardware and software innovations that will constitute a major qualitative upgrade in the organization, searchability, and public availability of the existing (and expanding) corpus. The enhancement and improved Web-accessibility of these corpora will be invaluable for linguistic research, enabling new kinds of discoveries and the testing of hypotheses that would otherwise have be difficult to investigate. On the computer vision side, the proposed new annotations will provide an extensive public dataset for training and benchmarking a variety of computer vision algorithms. This will facilitate research and expedite progress in gesture recognition, hand pose estimation, human tracking, and large vocabulary, and continuous ASL recognition. Furthermore, this dataset will be useful as training and benchmarking data for algorithms in the broader areas of computer vision, machine learning, and similarity-based indexing. The advances in linguistic knowledge about ASL and in computer-based ASL recognition that will be accelerated by the availability of resources of the kind proposed here will contribute to development of technologies for education and universal access. For example, tools for searching collections of ASL video for occurrences of specific signs, or converting ASL signing to English, are still far from attaining the level of functionality and usability to which users are accustomed for spoken/written languages. Our corpora will enable research that aims to bring such vision-based ASL recognition applications closer to reality. Moreover, these resources will afford important opportunities to individuals who would not otherwise be in a position to conduct such research (e.g., for lack of access to native ASL signers or high-quality synchronized video equipment, or lack of resources/expertise to carry out extensive linguistic annotations). Making our corpora available online will also allow the broader community of ASL users to access our data directly. Students of ASL will be able to retrieve video showing examples of a specific sign used in actual sentences, or examples of a grammatical construction. ASL instructors and teachers of the Deaf will also have easy access to video examples of lexical items and grammatical constructions as used by a variety of native signers for use in language instruction and evaluation. Thus, the proposed web interface to our data collection will be a useful educational resource for users, teachers, and learners of ASL.

该项目的目标是创建一个语言注释，公开可用，易于搜索的美国手语（ASL）视频语料库。这将构成一个重要的基础设施，使语言学和基于视觉的美国手语识别的新研究成为可能。此外，一个关键的目标是使该语料库易于更广泛的美国手语社区访问，包括美国手语的用户和学习者。由于我们的长期努力，我们有一个广泛的收集语言注释的视频数据来自美国手语的母语签署者。然而，这些语料库的潜在价值在很大程度上尚未开发，尽管它们被我们的团队和其他人广泛而有效地使用。现有的硬件和软件基础设施的限制使得搜索和识别感兴趣的数据，以及在我们的机构和其他研究人员之间共享数据变得非常麻烦。在这个项目中，我们提出了硬件和软件创新，这些创新将构成现有（和扩展）语料库的组织、可搜索性和公共可用性方面的重大质的升级。这些语料库的增强和网络可访问性的改进将对语言学研究具有不可估量的价值，使新的发现和假设的测试成为可能，否则很难进行调查。在计算机视觉方面，提出的新注释将为各种计算机视觉算法的训练和基准测试提供广泛的公共数据集。这将促进手势识别、手部姿势估计、人体跟踪、大词汇量和连续ASL识别方面的研究和进展。此外，该数据集将用于计算机视觉，机器学习和基于相似度的索引等更广泛领域的算法的训练和基准数据。关于美国手语的语言知识和基于计算机的美国手语识别的进步将会被这里提出的这种资源的可用性所加速，这将有助于教育技术的发展和普遍获取。例如，用于搜索特定符号出现的ASL视频集合，或将ASL签名转换为英语的工具，仍然远未达到用户习惯的口头/书面语言的功能和可用性水平。我们的语料库将使旨在使这种基于视觉的美国手语识别应用更接近现实的研究成为可能。此外，这些资源将为个人提供重要的机会，否则他们将无法进行此类研究（例如，缺乏获得母语ASL签字人或高质量同步视频设备的机会，或缺乏进行广泛语言注释的资源/专业知识）。将我们的语料库放到网上也将允许更广泛的美国手语用户社区直接访问我们的数据。美国手语的学生将能够检索视频，展示在实际句子中使用的特定符号的例子，或语法结构的例子。美国手语教师和聋人教师也可以很容易地获得各种母语手语使用的词汇项目和语法结构的视频示例，用于语言教学和评估。因此，我们所提出的数据收集的web界面将成为美国手语用户、教师和学习者的有用教育资源。