STTR Phase I: Dynamic Robust Hand Model for Gesture Intent Recognition

该项目更广泛的影响/商业潜力源于解决VR和AR行业基于手势的输入挑战，预计到2020年将增长到1500亿美元。Piper Jaffray将VR视为下一个大趋势，并估计到2025年VR市场价值将超过600亿美元。Piper Jaffray强调了将手和脚带入VR的外围设备的新市场机会。如果这项技术成功地降低了高技术风险，那么它代表了手势识别3D手部模型技术的巨大飞跃，并有可能成为AR、VR和3D应用的行业标准。我们公司将通过将该技术作为手模SDK授权给AR/VR和3D相机设备制造商和应用程序开发商，将高度交互的VR/AR和3D手势应用程序应用于游戏，娱乐，教育，医疗保健，设计，建筑，小企业技术转移研究（STTR）第一阶段项目开发了3D手势意图识别的突破性创新，可以在不同的3D相机，方向，位置和遮挡下稳健地工作。它解决了手势识别中的一个关键挑战，同时为虚拟和增强现实（VR/AR）以及3D深度相机支持的许多其他应用提供自然的空间交互。它解决了现有学术和商业手部模型面临的以下关键挑战，并涉及非常高的技术风险：1）在严重遮挡下的鲁棒性2）对视点变化的不变性3）低计算训练和跟踪复杂性4）对频繁手势/微手势序列的区分。我们将通过开发一种新的动态，强大的手部跟踪模型来解决这些问题，该模型的灵感来自计算机视觉社区不常用的机器学习技术。我们将通过开发以下目标来实现这一目标：1）使用训练的分类器生成手部姿势假设; 2）使用联合矩阵分解和完成的手部模型拟合; 3）手部模型的用户研究和评估。

The broader impact/commercial potential of this project stems from addressing the important hand gesture based input challenges of VR and AR industries that are expected to grow to $150B by 2020. Piper Jaffray identifies VR as the next mega trend and estimates the VR market to be worth more than $60B by 2025. Piper Jaffray highlights new market opportunities for peripheral devices that bring hands and feet into VR. This technology if successful in mitigating the high technical risks represents a huge leap in the state of the art in 3D hand models for gesture recognition and has the potential to be the industry standard for AR, VR and 3D applications. Our company will commercialize the project by licensing this technology as a hand model SDK to the AR/VR and 3D camera device makers and application developers to bring highly interactive VR/AR and 3D gesture applications to gaming, entertainment, education, healthcare, design, architecture, and manufacturing.This Small Business Technology Transfer Research (STTR) Phase I project develops a breakthrough innovation in 3D hand gesture intent recognition that can robustly work across different 3D cameras, orientations, positions and occlusions. It addresses a key challenge in gesture recognition while enabling natural spatial interactions for Virtual and Augmented Reality (VR/AR) and many other applications enabled by 3D depth cameras. It solves the following key challenges faced by existing academic and commercial hand models and involves very high technical risks: 1) robustness under heavy occlusions 2) invariance to view-point changes 3) low computational training and tracking complexity 4) discriminative to frequent gesture/micro-gesture sequences. We will tackle these by developing a novel dynamic, robust hand-tracking model inspired by a machine learning technique that is not commonly used by computer vision community. We will achieve this by developing the following objectives 1) hand pose hypothesis generation using trained classifiers 2) hand model fitting using joint matrix factorization and completion 3) user study and evaluation of the hand model.