STTR Phase I: Dynamic Robust Hand Model for Gesture Intent Recognition

该项目更广泛的影响/商业潜力来自于解决VR和AR行业基于手势的重要输入挑战，预计到2020年，这两个行业将增长到1500亿美元。Piper Jaffray认为虚拟现实是下一个大趋势，并估计到2025年虚拟现实市场的价值将超过600亿美元。Piper Jaffray强调了将手和脚带入VR的外围设备的新市场机会。如果这项技术成功地降低了高技术风险，代表着手势识别3D手部模型技术水平的巨大飞跃，并有可能成为AR、VR和3D应用的行业标准。我们公司将把这项技术作为手模SDK授权给AR/VR和3D摄像设备制造商和应用程序开发商，将这项技术商业化，将高度交互的VR/AR和3D手势应用程序带到游戏、娱乐、教育、医疗保健、设计、建筑和制造领域。这个小型企业技术转移研究(STTR)第一阶段项目开发了3D手势意图识别方面的突破性创新，可以跨不同的3D摄像头、方向、位置和遮挡稳健地工作。它解决了手势识别方面的一个关键挑战，同时实现了虚拟和增强现实(VR/AR)以及由3D深度相机实现的许多其他应用的自然空间交互。它解决了现有学术和商业手势模型所面临的以下关键挑战，并涉及到很高的技术风险：1)在严重遮挡下的健壮性；2)对视点变化的不变性；3)低计算训练和跟踪复杂度；4)对频繁手势/微手势序列的区分。我们将通过开发一种新的动态、健壮的手部跟踪模型来解决这些问题，该模型的灵感来自于计算机视觉社区不常用的机器学习技术。我们将通过以下目标来实现这一点：1)使用训练好的分类器生成手势假设；2)使用联合矩阵分解进行手部模型拟合；3)完成手部模型的用户学习和评估。

The broader impact/commercial potential of this project stems from addressing the important hand gesture based input challenges of VR and AR industries that are expected to grow to $150B by 2020. Piper Jaffray identifies VR as the next mega trend and estimates the VR market to be worth more than $60B by 2025. Piper Jaffray highlights new market opportunities for peripheral devices that bring hands and feet into VR. This technology if successful in mitigating the high technical risks represents a huge leap in the state of the art in 3D hand models for gesture recognition and has the potential to be the industry standard for AR, VR and 3D applications. Our company will commercialize the project by licensing this technology as a hand model SDK to the AR/VR and 3D camera device makers and application developers to bring highly interactive VR/AR and 3D gesture applications to gaming, entertainment, education, healthcare, design, architecture, and manufacturing.This Small Business Technology Transfer Research (STTR) Phase I project develops a breakthrough innovation in 3D hand gesture intent recognition that can robustly work across different 3D cameras, orientations, positions and occlusions. It addresses a key challenge in gesture recognition while enabling natural spatial interactions for Virtual and Augmented Reality (VR/AR) and many other applications enabled by 3D depth cameras. It solves the following key challenges faced by existing academic and commercial hand models and involves very high technical risks: 1) robustness under heavy occlusions 2) invariance to view-point changes 3) low computational training and tracking complexity 4) discriminative to frequent gesture/micro-gesture sequences. We will tackle these by developing a novel dynamic, robust hand-tracking model inspired by a machine learning technique that is not commonly used by computer vision community. We will achieve this by developing the following objectives 1) hand pose hypothesis generation using trained classifiers 2) hand model fitting using joint matrix factorization and completion 3) user study and evaluation of the hand model.