Enabling Interactive Perception Applications on Edge Devices

感知是指通过身体感觉对环境要素的感知。随着移动的和嵌入式设备的处理、通信和感测能力的增加，交互式感知应用越来越受欢迎。这些应用使用摄像头和其他高速传感器来执行感知任务，例如语音或物体识别，实现自然的人机界面和交互式增强现实体验，并为现场决策提供输入，例如控制自动驾驶汽车。交互式感知应用需要实时处理来自传感器的流数据，以提供及时的响应。由于边缘设备上的计算和存储资源有限，传统上，这些应用程序的部分或全部计算密集型组件都被卸载到云端。然而，这种方法不仅会从网络通信中引入不可忽略的额外延迟-在某些情况下，使应用程序无法响应，而且还会通过将用户数据上传到远程服务器并将其存储在远程服务器上来引起隐私问题。我的研究计划的长期目标是协同设计网络嵌入式和移动的系统的计算模型。短期目标是通过克服资源障碍，在边缘设备上实现交互式感知应用。为子索短期目的，我们计划在五年期间进行四项主要研究活动，即i）对边缘设备上交互式应用程序的性能瓶颈和功率效率进行基准测试，ii）为协作边缘设备开发分布式模型，iii）设计模型缓存算法，以及iv）运行时配置优化。共有7名HQP（2名博士，1名硕士和5名本科研究生）将通过该计划的持续时间进行培训。该研究处于移动的计算、网络和机器学习的交叉点。它将使人们更好地了解交互式感知应用的系统考虑因素和机器学习模型之间的相互作用。拟议的研究计划将提供HQP在学术界和工业界高度重视的技能方面的巨大培训机会。

Perception means awareness of the elements of environment through physical sensation. With increasing processing, communication and sensing capabilities of mobile and embedded devices, interactive perception applications are gaining popularity. These applications use cameras and other high-rate sensors to perform perception tasks, such as speech or object recognition, enable natural human-machine interfaces and interactive augmented-reality experiences, and provide inputs to in-situ decision making such as controlling an autonomous vehicle. Interactive perception applications require real-time processing of streaming data from sensors to provide timely response. Due to limited compute and storage resources on edge devices, traditionally, part or all of the compute-intensive components of such applications are offloaded to the cloud. However, such approaches not only introduce non-negligible extra latency from network communication -- in some cases, rendering the application unresponsive, but also pose privacy concerns by uploading to and storing user data on remote servers. The long-term goal of my research program is to co-design computation models with networked embedded and mobile systems. The short-term goal is to enable interactive perception applications on edge devices by overcoming their resource barriers. To realize the short- term goal, we plan to pursue four main research activities in the five-year period, namely, i) benchmarking performance bottleneck and power efficiency of interactive applications on edge devices, ii) developing distributed models for cooperating edge devices, iii) designing model caching algorithms, and iv) run- time configuration optimization. A total of seven HQP (2 PhDs, 1 Master's and 5 undergraduate research students) will be trained through the duration of the program. The research is at the intersection of mobile computing, networking and machine learning. It will result in a better understanding of the interplay between systems considerations and machine learning models for interactive perception applications. The proposed research program will offer HQP great training opportunities in skillsets highly valued in both academia and industry.