CAREER: Large Vocabulary Gesture Recognition for Everyone: Gesture Modeling and Recognition Tools for System Builders and Users

职业：适合所有人的大词汇量手势识别：面向系统构建者和用户的手势建模和识别工具

• 批准号: 1055062
• 负责人: Vassilis Athitsos
• 金额: $ 51.36万
• 依托单位: University of Texas at Arlington
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-04-01 至 2017-03-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1055062&HistoricalAwards=false
• 关键词: CAREER; Large; Vocabulary; Gesture; Recognition

The PI's goal in this project is to develop new methods for automatically annotating, recognizing, and indexing large vocabularies of gestures, and to use these methods to create an integrated set of tools for sign language recognition. Current state-of-the-art methods for recognizing large vocabularies of gestures have significant limitations that impact both system design and the user experience. Many methods assume the existence of a near-perfect hand detector/tracker; that is a limiting assumption, which prevents deployment of these methods in complex real-world settings where such accuracy is unachievable. In the absence of perfect hand detectors, system design may involve a large investment in manual annotation of training videos (e.g., specifying hand locations), so as to provide sufficiently clean information to training modules. The user experience is affected by the limited accuracy and robustness of existing applications. In this research the PI will address these issues by explicitly designing recognition and indexing methods that require neither perfect hand detectors nor extensive manual annotations, thus making it substantially easier to deploy accurate and efficient gesture recognition systems in real-world settings. The PI will achieve these objectives through theoretical advances in the current state of the art in computer vision, pattern recognition, and database indexing. The unifying theme in the project is the integration of low-level tracking modules that produce imperfect output, with recognition and indexing methods that are designed to take as input this imperfect output from the tracking modules. Novel articulated tracking methods will be developed that utilize probabilistic graph models to provide fully automatic long-term tracking, while improving upon the excessive time complexity that probabilistic graph models currently incur. New methods will be designed for extracting and exploiting information from hand appearance. As these novel modeling and recognition methods will violate standard assumptions made by existing indexing methods, new indexing methods will be formulated which will improve the efficiency of search in large databases of dynamic gestures and static hand shapes within the proposed framework.Broader Impacts: Project outcomes will significantly improve the ability of sign language users around the world to search databases of sign language videos and to perform tasks such as looking up the meaning of an unknown sign or retrieving occurrences of a sign of interest in videos of continuous signing. These search tools will have an impact in educational settings, facilitating both learning a sign language and accessing arbitrary information available in a sign language. To these ends, the PI will make his software freely available to the public online. He will also work with experts in American Sign Language to implement key applications using his tools, which will be made available to Deaf students. The PI will furthermore develop a publicly available package of gesture recognition source code, applications, and datasets that will help student researchers at all levels engage in gesture recognition research. As an additional outreach activity intended to attract young people to careers in science, the PI will co-organize summer camps that educate junior high and high school students in computer science.

PI在该项目中的目标是开发新的方法，以自动注释，识别和索引大的手势词汇，并使用这些方法来创建一组集成的工具，以识别手势语言。 当前识别大型手势词汇的最新方法具有重大限制，从而影响系统设计和用户体验。 许多方法都假定存在近乎完美的手探测器/跟踪器。这是一个有限的假设，它阻止了这些方法在这种准确性无法实现的复杂现实世界中的部署。 在没有完美的手探测器的情况下，系统设计可能涉及大量投资培训视频的手动注释（例如，指定手部位置），以便为培训模块提供足够清洁的信息。 用户体验受现有应用程序的准确性和鲁棒性有限的影响。 在这项研究中，PI将通过明确设计识别和索引方法来解决这些问题，这些方法既不需要完美的手动检测器也不需要大量的手动注释，从而使在现实世界中部署准确有效的手势识别系统变得更加容易。 PI将通过计算机视觉，模式识别和数据库索引的当前技术状态中的理论进步实现这些目标。 该项目中的统一主题是集成低级跟踪模块，这些模块产生了不完善的输出，并采用识别和索引方法，这些方法旨在将其作为输入跟踪模块的输出。 将开发出新的清晰跟踪方法，利用概率图模型提供全自动的长期跟踪，同时改善了概率图模型当前产生的过度时间复杂性。 新方法将设计用于从手外观中提取和利用信息。 由于这些新颖的建模和识别方法将违反现有索引方法做出的标准假设，因此将提出新的索引方法，这将提高搜索效率的搜索效率，并在拟议的框架内进行动态姿态和静态手动形状，在拟议的框架内进行静态的手段。Boader的影响：项目量之间的搜索量很大，可以在搜索范围内的含义，并在范围内表现出符号的含义，以提高标志性的含义，以示为标志的能力，以便在sign语言的能力中构成同一同步的能力。或检索在连续签名视频中有兴趣的迹象。 这些搜索工具将在教育环境中产生影响，既有助于学习手语和访问用手语的任意信息。 对于这些目的，PI将使他的软件在线免费提供给公众。 他还将与美国手语专家合作，使用他的工具实施关键应用程序，这将为聋哑学生提供。 PI还将开发出公开可用的手势识别源代码，应用程序和数据集的包装，以帮助各级学生研究人员从事手势识别研究。 作为旨在吸引年轻人从事科学职业的额外外展活动，PI将共同组织夏令营，以教育初中和高中生的计算机科学。