CSR: Small:Collaborative Research: Decentralized Real-Time Machine Learning Systems on Near-User Edge Devices

越来越多的物联网（IoT）设备产生了大量需要实时处理和分析的原始数据。由于执行计算成本高的任务，如计算机视觉和自然语言处理，对于物联网设备来说通常是一个挑战，因此它们的大部分计算目前都卸载到云服务器上。然而，这种卸载会增加隐私风险以及对网络连接的依赖。为了解决这一挑战，该项目利用已经连接的物联网设备的分布式计算能力，实时执行高计算能力的应用。该项目由三个任务组成。首先是为多个物联网设备开发分布式机器学习（ML）系统。该项目将包括研究如何在具有可靠连接的节点之间进行通信，以及如何在运行时以很小的开销动态更改每个节点的任务。第二是最优任务分配和调度算法的发展。在这里，将使用机器学习方法为每个分布式系统配置生成最优的识别模型架构。第三是低分辨率深度神经网络（DNN）系统的发展，以利用低功耗计算节点。这些深度神经网络系统的开发将涉及识别适合不同配置的多个低分辨率滤波器。提出的技术工作将推进并行和分散DNN系统实施的最新技术，从而使依赖计算的所有科学领域受益。分散式DNN系统将为监控和汽车等应用的功率受限移动平台提供新的机会。研究结果将导致计算机体系结构和系统的新材料/课程。拟议的基础设施也将用于指导本科生的研究活动。软件基础设施将作为开源项目维护，可以在https://github.com/parallel-ml上找到。当有新的结果可用时，将定期更新。研究结果将发表在会议、期刊和技术报告上。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The ever-increasing number of Internet of Things (IoT) devices generate large quantities of raw data that need to be processed and analyzed in real time. Since conducting computationally expensive tasks, such as computer vision and natural language processing, is often a challenge for IoT devices, most of their computations are currently offloaded to cloud servers. However, this offloading leads to an increased risk for privacy as well as a dependency on network connectivity. To solve this challenge, the project utilizes the distributed computing power of already connected IoT devices to perform high computing power applications in real time.The project is composed of three tasks. First is the development of distributed machine learning (ML) systems for multiple IoT devices. The project will involve studying how to communicate between nodes with reliable connections and how to dynamically change the job of each node at run-time with little overhead. Second is the development of optimal task assignment and scheduling algorithms. Here, a machine learning approach will be used to generate a recognition model architecture optimal for each distributed system configuration. Third is the development of low-resolution deep neural network (DNN) systems to utilize low-power computing nodes. The development of these DNN systems will involve identifying multiple low-resolution filters that are optimal for varying configurations.The proposed technical work will advance the state of the art in implementation of parallel and decentralized DNN systems, thereby benefiting all scientific fields of endeavor that rely on computing. The decentralized DNN system will offer new opportunities in power constrained mobile platforms for applications including surveillance and automotive. The research results will lead to new materials/courses for computer architecture and systems. The proposed infrastructure will also be used to guide undergraduate students' research activities. The software infrastructure will be maintained as an open source project, which can be found at https://github.com/parallel-ml. It will be updated periodically as new outcomes become available. The results will be published in conferences, journals and technical reports.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.