CNS Core: Small: Toward Opportunistic, Fast, and Robust In-Cache AI Acceleration at the Edge

到2025年，物联网(IoT)预计将包含约750亿台互联设备。许多支持人工智能(AI)的物联网设备由感官成像系统组成，可以从人和环境中收集数据。然而，智能手机、可穿戴设备等小型物联网设备计算能力不足，以及内存/计算密集型AI任务，阻碍了AI技术在此类设备中的广泛部署。这一建议实现了从最先进的以云为中心的物联网方法到以数据为中心的方法的平稳过渡，使移动边缘设备能够通过将缓存重新用于数据并行处理单元，在传感器附近执行计算。这将显著降低数据传输到云的功耗和延迟。此外，该项目寻求在边缘设备中设计和部署新的面向硬件的AI算法，以实现高效的图像处理，在保持准确性的同时降低计算复杂性和内存访问。通过该项目开发的技术，可以引入更强大和更稳定的物联网设备，确保加快运营，应用范围涵盖医疗监控、汽车应用、工农业传感、智能基础设施等。将采用全面的电路到系统评估框架，在几个物联网工作负载套件上系统地评估物联网系统的性能。该项目将大力开发本科生和研究生课程模块，宣传可移植和开源模式，并通过出版物/在会议和研讨会上的演讲以及让本科生参与STEM来扩大STEM的参与。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Internet of Things (IoT) is anticipated to comprise approximately 75+ billion interconnected devices by 2025. Many Artificial Intelligence (AI)-enabled IoT devices consist of sensory imaging systems that enable data collection from people and the environment. However, insufficient computing ability of small IoT devices such as smartphones, wearable devices, etc., and memory/compute-intensive AI tasks prevent AI techniques from being widely deployed in such devices. This proposal enables a smooth transition from the state-of-the-art cloud-centric IoT approach to a data-centric approach, enabling mobile edge devices to perform computation close to the sensor by repurposing the cache memory to a data-parallel processing unit. This will remarkably reduce the power consumption and latency of data transmission to the cloud. Moreover, this project seeks to design and deploy new hardware-oriented AI algorithms into edge devices for efficient image processing, reducing the computation complexity and memory access, while maintaining accuracy. With the technologies developed in this project, more powerful and stable IoT devices can be introduced ensuring accelerated operation with applications of societal importance spanning healthcare monitoring, automotive applications, industrial and agriculture sensing, intelligent infrastructure, etc. A comprehensive circuit-to-system assessment framework will be adopted to systematically evaluate the performance of the system on several IoT workload suites. This project will make a strong effort on developing undergraduate and graduate course modules, propagating transportable and open-source models, and broadening STEM participation through publications/presentations at conferences and workshops and involving undergraduate minority students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.