CAREER: Large Vocabulary Gesture Recognition for Everyone: Gesture Modeling and Recognition Tools for System Builders and Users

职业：适合所有人的大词汇量手势识别：面向系统构建者和用户的手势建模和识别工具

• 批准号: 1055062
• 负责人: Vassilis Athitsos
• 金额: $ 51.36万
• 依托单位: University of Texas at Arlington
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-04-01 至 2017-03-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1055062&HistoricalAwards=false
• 关键词: CAREER; Large; Vocabulary; Gesture; Recognition

The PI's goal in this project is to develop new methods for automatically annotating, recognizing, and indexing large vocabularies of gestures, and to use these methods to create an integrated set of tools for sign language recognition. Current state-of-the-art methods for recognizing large vocabularies of gestures have significant limitations that impact both system design and the user experience. Many methods assume the existence of a near-perfect hand detector/tracker; that is a limiting assumption, which prevents deployment of these methods in complex real-world settings where such accuracy is unachievable. In the absence of perfect hand detectors, system design may involve a large investment in manual annotation of training videos (e.g., specifying hand locations), so as to provide sufficiently clean information to training modules. The user experience is affected by the limited accuracy and robustness of existing applications. In this research the PI will address these issues by explicitly designing recognition and indexing methods that require neither perfect hand detectors nor extensive manual annotations, thus making it substantially easier to deploy accurate and efficient gesture recognition systems in real-world settings. The PI will achieve these objectives through theoretical advances in the current state of the art in computer vision, pattern recognition, and database indexing. The unifying theme in the project is the integration of low-level tracking modules that produce imperfect output, with recognition and indexing methods that are designed to take as input this imperfect output from the tracking modules. Novel articulated tracking methods will be developed that utilize probabilistic graph models to provide fully automatic long-term tracking, while improving upon the excessive time complexity that probabilistic graph models currently incur. New methods will be designed for extracting and exploiting information from hand appearance. As these novel modeling and recognition methods will violate standard assumptions made by existing indexing methods, new indexing methods will be formulated which will improve the efficiency of search in large databases of dynamic gestures and static hand shapes within the proposed framework.Broader Impacts: Project outcomes will significantly improve the ability of sign language users around the world to search databases of sign language videos and to perform tasks such as looking up the meaning of an unknown sign or retrieving occurrences of a sign of interest in videos of continuous signing. These search tools will have an impact in educational settings, facilitating both learning a sign language and accessing arbitrary information available in a sign language. To these ends, the PI will make his software freely available to the public online. He will also work with experts in American Sign Language to implement key applications using his tools, which will be made available to Deaf students. The PI will furthermore develop a publicly available package of gesture recognition source code, applications, and datasets that will help student researchers at all levels engage in gesture recognition research. As an additional outreach activity intended to attract young people to careers in science, the PI will co-organize summer camps that educate junior high and high school students in computer science.

PI在这个项目中的目标是开发新的方法来自动标注、识别和索引大量手势词汇，并使用这些方法来创建一套集成的手语识别工具。当前用于识别大型手势词汇表的最新方法具有显著的局限性，影响了系统设计和用户体验。许多方法假设存在近乎完美的手部检测器/跟踪器；这是一个限制性的假设，这阻止了这些方法在无法实现这种准确性的复杂的现实世界环境中的部署。在没有完美的手部检测器的情况下，系统设计可能涉及对训练视频的人工注释(例如，指定手部位置)的大量投资，以便向训练模块提供足够干净的信息。现有应用程序的准确性和健壮性有限，影响了用户体验。在这项研究中，PI将通过明确设计识别和索引方法来解决这些问题，这些方法既不需要完美的手部检测器，也不需要大量的手动注释，从而使在现实世界中部署准确和高效的手势识别系统变得更加容易。PI将通过在计算机视觉、模式识别和数据库索引方面的当前技术水平的理论进步来实现这些目标。该项目的统一主题是将产生不完美输出的低级跟踪模块与识别和索引方法相结合，这些方法旨在将跟踪模块的这一不完美输出作为输入。将开发新的铰接式跟踪方法，其利用概率图模型来提供全自动的长期跟踪，同时改进概率图模型目前引起的过高的时间复杂性。将设计新的方法来提取和利用手部外观的信息。由于这些新的建模和识别方法将违反现有索引方法所做的标准假设，因此将制定新的索引方法，以提高在建议框架内的动态手势和静态手形大型数据库中的搜索效率。广泛影响：项目成果将显著提高世界各地手语用户搜索手语视频数据库的能力，并执行诸如查找未知手语的含义或在连续手语视频中检索感兴趣的手语的出现次数等任务。这些搜索工具将对教育环境产生影响，既便于学习手语，也便于获取手语中的任意信息。为此，私家侦探将在网上免费向公众提供他的软件。他还将与美国手语专家合作，使用他的工具实现关键应用程序，并向聋人学生提供这些应用程序。PI将进一步开发一套公开可用的手势识别源代码、应用程序和数据集，以帮助所有级别的学生研究人员从事手势识别研究。作为一项旨在吸引年轻人投身科学职业的额外外联活动，PI将共同组织夏令营，对初中生和高中生进行计算机科学教育。