Hand Segmentation and Detection Applied to Mobile Devices Using a Noisy Input Depth Image Stream****

点击翻译按钮获取中文摘要

Human computer interaction is rapidly evolving with an emerging focus on gesture recognition tools. There are several state-of-the-art algorithms that tackle aspects of this problem, but no convincing result has been presented that tracks motion, localizes hands, and interprets gestures. Leap Motion specializes in hand tracking but does not produce suitable gesture detection and interpretation. Mobile devices such as iPhone are equipped with depth cameras, but the industry needs innovative, improved algorithms to produce an accurate 3D representation of a moving hand. Hence, accurate real-time human hand tracking represents a fertile area of advanced research and technology development, with no robust solution to date.** Toronto-based Xesto has developed a cloud-based machine learning platform to recognize and record gestures for use in gesture-based applications using depth map from Leap Motion and mobile devices with depth cameras. Xesto has built a prototype that currently requires a superior 3D representation of the hand, while leveraging robust hand detection algorithms on Xesto's device agnostic platform.** This project represents a collaboration between Professor Deepa Kundur and Xesto to improve the real time performance of hand tracking on mobile devices. Specifically, the project makes use of Xesto's RGB-D hand-pose detection and gesture recognition pipeline that is applicable to any depth sensor. The research aims to design a novel convolutional neural network architecture that uses temporal information to produce an accurate real-time heat map of hand-features. This problem has a high level of technical depth in estimating and optimizing the energy function that constructs articulable objects with large degrees of freedom, self-similar parts and is subject to self-occlusion. ** The proposed research is expected to have impact in the field of hand tracking providing Xesto a technological advantage to develop superior applications for health, retail and automotive industry. Further, this work will grant each mobile device user a new way to interact with their machines only through gestures.