Hand Segmentation and Detection Applied to Mobile Devices Using a Noisy Input Depth Image Stream****

人机交互正在迅速发展，手势识别工具成为关注的焦点。有几种最先进的算法可以解决这个问题的各个方面，但还没有令人信服的结果来跟踪运动，定位手部和解释手势。Leap Motion专注于手部跟踪，但无法产生合适的手势检测和解释。iPhone等移动的设备配备了深度摄像头，但该行业需要创新的改进算法来生成移动手部的准确3D表示。因此，准确的实时人手跟踪代表了先进研究和技术开发的肥沃领域，迄今为止还没有强大的解决方案。 总部位于多伦多的Xesto开发了一个基于云的机器学习平台，可以识别和记录手势，用于使用Leap Motion的深度图和带有深度摄像头的移动的设备的基于手势的应用。Xesto已经建立了一个原型，目前需要一个上级3D表示的手，同时利用强大的手检测算法在Xesto的设备不可知的平台。** 该项目代表了Deepa Kundur教授和Xesto之间的合作，以提高移动的设备上手部跟踪的真实的时间性能。具体来说，该项目利用了Xesto的RGB-D手势检测和手势识别管道，适用于任何深度传感器。该研究旨在设计一种新的卷积神经网络架构，该架构使用时间信息来生成手部特征的准确实时热图。这个问题在估计和优化能量函数方面具有很高的技术深度，所述能量函数构造具有大自由度、自相似部分并且受到自遮挡的可铰接对象。** 预计拟议的研究将在手部跟踪领域产生影响，为Xesto提供技术优势，为健康，零售和汽车行业开发上级应用。 此外，这项工作将授予每个移动终端用户一种新的方式来与他们的机器只通过手势进行交互。

Human computer interaction is rapidly evolving with an emerging focus on gesture recognition tools. There are several state-of-the-art algorithms that tackle aspects of this problem, but no convincing result has been presented that tracks motion, localizes hands, and interprets gestures. Leap Motion specializes in hand tracking but does not produce suitable gesture detection and interpretation. Mobile devices such as iPhone are equipped with depth cameras, but the industry needs innovative, improved algorithms to produce an accurate 3D representation of a moving hand. Hence, accurate real-time human hand tracking represents a fertile area of advanced research and technology development, with no robust solution to date.** Toronto-based Xesto has developed a cloud-based machine learning platform to recognize and record gestures for use in gesture-based applications using depth map from Leap Motion and mobile devices with depth cameras. Xesto has built a prototype that currently requires a superior 3D representation of the hand, while leveraging robust hand detection algorithms on Xesto's device agnostic platform.** This project represents a collaboration between Professor Deepa Kundur and Xesto to improve the real time performance of hand tracking on mobile devices. Specifically, the project makes use of Xesto's RGB-D hand-pose detection and gesture recognition pipeline that is applicable to any depth sensor. The research aims to design a novel convolutional neural network architecture that uses temporal information to produce an accurate real-time heat map of hand-features. This problem has a high level of technical depth in estimating and optimizing the energy function that constructs articulable objects with large degrees of freedom, self-similar parts and is subject to self-occlusion. ** The proposed research is expected to have impact in the field of hand tracking providing Xesto a technological advantage to develop superior applications for health, retail and automotive industry. Further, this work will grant each mobile device user a new way to interact with their machines only through gestures.