CNS Core: Small: Toward Opportunistic, Fast, and Robust In-Cache AI Acceleration at the Edge

到2025年，物联网（IoT）预计将包括约750多亿个互联设备。许多支持人工智能（AI）的物联网设备由传感成像系统组成，可以从人和环境中收集数据。然而，智能手机、可穿戴设备等小型物联网设备的计算能力不足，以及内存/计算密集型AI任务，阻碍了AI技术在此类设备中的广泛部署。该提案实现了从最先进的以云为中心的物联网方法到以数据为中心的方法的平稳过渡，使移动边缘设备能够通过将缓存内存重新利用为数据并行处理单元来执行靠近传感器的计算。这将显著降低功耗和数据传输到云的延迟。此外，该项目旨在将新的面向硬件的人工智能算法设计和部署到边缘设备中，以实现高效的图像处理，降低计算复杂性和内存访问，同时保持准确性。通过该项目开发的技术，可以引入更强大、更稳定的物联网设备，确保加速运行，并在医疗监控、汽车应用、工农业传感、智能基础设施等具有社会重要性的应用中发挥作用。将采用全面的电路到系统评估框架，系统地评估系统在多个物联网工作负载套件上的性能。该项目将大力开发本科和研究生课程模块，推广可移植和开源模型，并通过在会议和研讨会上发表出版物/演讲以及让本科少数民族学生参与，扩大STEM的参与。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Internet of Things (IoT) is anticipated to comprise approximately 75+ billion interconnected devices by 2025. Many Artificial Intelligence (AI)-enabled IoT devices consist of sensory imaging systems that enable data collection from people and the environment. However, insufficient computing ability of small IoT devices such as smartphones, wearable devices, etc., and memory/compute-intensive AI tasks prevent AI techniques from being widely deployed in such devices. This proposal enables a smooth transition from the state-of-the-art cloud-centric IoT approach to a data-centric approach, enabling mobile edge devices to perform computation close to the sensor by repurposing the cache memory to a data-parallel processing unit. This will remarkably reduce the power consumption and latency of data transmission to the cloud. Moreover, this project seeks to design and deploy new hardware-oriented AI algorithms into edge devices for efficient image processing, reducing the computation complexity and memory access, while maintaining accuracy. With the technologies developed in this project, more powerful and stable IoT devices can be introduced ensuring accelerated operation with applications of societal importance spanning healthcare monitoring, automotive applications, industrial and agriculture sensing, intelligent infrastructure, etc. A comprehensive circuit-to-system assessment framework will be adopted to systematically evaluate the performance of the system on several IoT workload suites. This project will make a strong effort on developing undergraduate and graduate course modules, propagating transportable and open-source models, and broadening STEM participation through publications/presentations at conferences and workshops and involving undergraduate minority students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.