Collaborative Research: CI-ADDO-EN: Development of Publicly Available, Easily Searchable, Linguistically Analyzed, Video Corpora for Sign Language and Gesture Research

该项目的目标是创建一个语言注释，公开可用，易于搜索的美国手语（ASL）视频语料库。这将构成一个重要的基础设施，使新的研究在语言学和基于视觉的识别美国手语。此外，一个关键的目标是使这个语料库容易访问更广泛的美国手语社区，包括美国手语的用户和学习者。由于我们的长期努力，我们有一个广泛的语言注释的视频数据从美国手语的本地签名收集。然而，这些语料库的潜在价值在很大程度上尚未得到开发，尽管我们的团队和其他人广泛而富有成效地使用它们。我们的硬件和软件基础设施中的现有限制使得搜索和识别感兴趣的数据以及在我们的机构之间和与其他研究人员共享数据变得非常繁琐。在这个项目中，我们提出了硬件和软件的创新，将构成一个重大的质量升级的组织，搜索能力，和现有的（和扩大）语料库的公共可用性。这些语料库的增强和改进的网络可访问性将是非常宝贵的语言学研究，使新的发现和假设的测试，否则将难以调查。在计算机视觉方面，拟议的新注释将提供广泛的公共数据集，用于训练和基准测试各种计算机视觉算法。这将促进手势识别、手部姿势估计、人体跟踪、大词汇量和连续ASL识别的研究和加速进展。此外，该数据集将作为计算机视觉、机器学习和基于相似性的索引等更广泛领域的算法的训练和基准数据。关于美国手语的语言知识和基于计算机的美国手语识别方面的进展将因可获得此处提议的资源而加速，这将有助于发展教育和普及技术。例如，用于搜索ASL视频集合以寻找特定标志的出现或将ASL标志转换为英语的工具仍然远远没有达到用户习惯于口语/书面语言的功能和可用性水平。我们的语料库将使旨在使这种基于视觉的ASL识别应用更接近现实的研究成为可能。此外，这些资源将为那些原本无法进行此类研究的个人提供重要的机会（例如，由于无法访问本地ASL签名者或高质量同步视频设备，或缺乏资源/专业知识来执行广泛的语言注释）。在线提供我们的语料库也将允许更广泛的ASL用户社区直接访问我们的数据。ASL的学生将能够检索视频显示在实际句子中使用的特定符号的例子，或语法结构的例子。聋人的美国手语教师和教师也将很容易获得词汇项目和语法结构的视频例子，这些例子被各种本地签名者用于语言教学和评估。因此，建议的网络接口，我们的数据收集将是一个有用的教育资源，为用户，教师和学习者的美国手语。

The goal of this project is to create a linguistically annotated, publicly available, and easily searchable corpus of video from American Sign Language (ASL). This will constitute an important piece of infrastructure, enabling new kinds of research in both linguistics and vision-based recognition of ASL. In addition, a key goal is to make this corpus easily accessible to the broader ASL community, including users and learners of ASL. As a result of our long-term efforts, we have an extensive collection of linguistically annotated video data from native signers of ASL. However, the potential value of these corpora has been largely untapped, notwithstanding their extensive and productive use by our team and others. Existing limitations in our hardware and software infrastructure make it cumbersome to search and identify data of interest, and to share data among our institutions and with other researchers. In this project, we propose hardware and software innovations that will constitute a major qualitative upgrade in the organization, searchability, and public availability of the existing (and expanding) corpus. The enhancement and improved Web-accessibility of these corpora will be invaluable for linguistic research, enabling new kinds of discoveries and the testing of hypotheses that would otherwise have be difficult to investigate. On the computer vision side, the proposed new annotations will provide an extensive public dataset for training and benchmarking a variety of computer vision algorithms. This will facilitate research and expedite progress in gesture recognition, hand pose estimation, human tracking, and large vocabulary, and continuous ASL recognition. Furthermore, this dataset will be useful as training and benchmarking data for algorithms in the broader areas of computer vision, machine learning, and similarity-based indexing. The advances in linguistic knowledge about ASL and in computer-based ASL recognition that will be accelerated by the availability of resources of the kind proposed here will contribute to development of technologies for education and universal access. For example, tools for searching collections of ASL video for occurrences of specific signs, or converting ASL signing to English, are still far from attaining the level of functionality and usability to which users are accustomed for spoken/written languages. Our corpora will enable research that aims to bring such vision-based ASL recognition applications closer to reality. Moreover, these resources will afford important opportunities to individuals who would not otherwise be in a position to conduct such research (e.g., for lack of access to native ASL signers or high-quality synchronized video equipment, or lack of resources/expertise to carry out extensive linguistic annotations). Making our corpora available online will also allow the broader community of ASL users to access our data directly. Students of ASL will be able to retrieve video showing examples of a specific sign used in actual sentences, or examples of a grammatical construction. ASL instructors and teachers of the Deaf will also have easy access to video examples of lexical items and grammatical constructions as used by a variety of native signers for use in language instruction and evaluation. Thus, the proposed web interface to our data collection will be a useful educational resource for users, teachers, and learners of ASL.