MicroEdge: a multi-tenant edge cluster system architecture for scalable camera processing
MicroEdge: a multi-tenant edge cluster system architecture for scalable camera processing
复制标题
DOI：
10.1145/3528535.3565254
复制
发表时间：
2022-11
期刊：
影响因子：
--
通讯作者：
Difei Cao;Jinsun Yoo;Zhuangdi Xu;Enrique Saurez;Harshit Gupta;T. Krishna;U. Ramachandran
中科院分区：
文献类型：
--
作者：
Difei Cao;Jinsun Yoo;Zhuangdi Xu;Enrique Saurez;Harshit Gupta;T. Krishna;U. Ramachandran
随着高带宽相机和AR/VR设备的激增，以及它们在态势感知应用中的使用越来越多，边缘计算在满足此类应用的吞吐量要求方面越来越重要。这项工作的重点是对相机帧执行实时机器学习推理的相机应用程序。我们发现，基于机器学习的相机应用程序由于模型未充分利用或过度利用加速器而遭受硬件资源碎片化。同时，支持TPU等加速器的细粒度资源共享具有挑战性，因为它们只能以运行到完成的方式顺序处理请求。我们提出了MicroEdge，一个多租户低成本边缘集群，用于在边缘运行的相机处理应用程序。MicroEdge通过扩展K3 s（一种特定于边缘的Kubernetes发行版）为Coral TPU提供多租户支持。通过准入控制算法，它允许TPU资源的部分分配与应用程序流水线要求相称，以确保TPU得到充分利用。使用实时相机处理应用程序和真实世界的跟踪，我们表明，MicroEdge可以支持高达2.8倍的相机流为给定的硬件配置相比，香草K3，同时保持可扩展性和性能要求。
With the proliferation of high bandwidth cameras and AR/VR devices, and their increasing use in situation awareness applications, edge computing is gaining prominence to meet the throughput requirements of such applications. This work focuses on camera applications that perform real-time Machine Learning inferences on camera frames. We find that Machine Learning based camera applications suffer from hardware resource fragmentation due to models under-utilizing or over-utilizing the accelerator. Meanwhile, it is challenging to support fine-grained resource sharing for accelerators such as TPUs because they can only process requests sequentially in a run to completion fashion. We present MicroEdge, a multi-tenant low-cost edge cluster for camera processing applications running at the edge. MicroEdge provides multi-tenancy support for Coral TPUs by extending K3s, an edge-specific distribution of Kubernetes. Through an admission control algorithm, it allows for fractional assignment of TPU resources commensurate with the application pipeline requirements to ensure that the TPUs are fully utilized. Using real-time camera processing applications and a real-world trace, we show that MicroEdge can support up to 2.8x camera streams for a given hardware configuration compared to vanilla K3s, while maintaining scalability and performance requirements.