Enabling Interactive Perception Applications on Edge Devices

点击翻译按钮获取中文摘要

Perception means awareness of the elements of environment through physical sensation. With increasing processing, communication and sensing capabilities of mobile and embedded devices, interactive perception applications are gaining popularity. These applications use cameras and other high-rate sensors to perform perception tasks, such as speech or object recognition, enable natural human-machine interfaces and interactive augmented-reality experiences, and provide inputs to in-situ decision making such as controlling an autonomous vehicle. Interactive perception applications require real-time processing of streaming data from sensors to provide timely response. Due to limited compute and storage resources on edge devices, traditionally, part or all of the compute-intensive components of such applications are offloaded to the cloud. However, such approaches not only introduce non-negligible extra latency from network communication -- in some cases, rendering the application unresponsive, but also pose privacy concerns by uploading to and storing user data on remote servers. The long-term goal of my research program is to co-design computation models with networked embedded and mobile systems. The short-term goal is to enable interactive perception applications on edge devices by overcoming their resource barriers. To realize the short- term goal, we plan to pursue four main research activities in the five-year period, namely, i) benchmarking performance bottleneck and power efficiency of interactive applications on edge devices, ii) developing distributed models for cooperating edge devices, iii) designing model caching algorithms, and iv) run- time configuration optimization. A total of seven HQP (2 PhDs, 1 Master's and 5 undergraduate research students) will be trained through the duration of the program. The research is at the intersection of mobile computing, networking and machine learning. It will result in a better understanding of the interplay between systems considerations and machine learning models for interactive perception applications. The proposed research program will offer HQP great training opportunities in skillsets highly valued in both academia and industry.