CAREER: Robust and Ultra-low-power Spatial Intelligence

职业：稳健且超低功耗的空间智能

• 批准号: 2046435
• 负责人: Amit Trivedi
• 金额: $ 56.08万
• 依托单位: University of Illinois at Chicago
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-05-01 至 2026-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2046435&HistoricalAwards=false
• 关键词: CAREER; Robust; Ultra; low; power

Machine learning-based autonomous navigation presents a unique opportunity for electronic sensors such as cameras to proactively explore their application spaces. For example, an autonomous flying camera can locate infected plants in an agriculture field to prevent the disease spread. Similarly, in an industrial plant, self-flying gas sensors can swiftly identify gas leaks. Most electronic sensors, thus enlivened by machine learning-based self-navigation, can have dramatically elevated use-cases. For practicality, however, the flying vehicle (i.e., drone) must be small enough to be inconspicuous and non-intrusive to users, such as people in offices. The small size of the drones is also necessary to navigate through constricted spaces. Since a tiny drone can only carry a tiny battery's payload, minimizing power dissipation for onboard processing of machine learning-based navigation is quite critical. Furthermore, flying space can be highly dynamic, e.g., there will be a movement of people and changes in lighting conditions in indoor applications. Therefore, drone's navigation must be resilient against these factors. This research is expected to develop new hardware implementations for machine learning-based autonomous navigation that can sustain on a tiny battery. The new hardware will also have a minimal footprint for easy integration with tiny drones. The platform will also robustly handle real-world's uncertainties, such as changes in indoor lighting conditions and people's movement. The investigator will also pursue various synergistic educational activities such as organizing workshops on machine learning at local high schools, developing a new course on machine learning hardware, and mentoring undergraduate students through this research. The investigator will specifically develop a platform for deep learning-based continuous tracking of drone's position and orientation. The platform will operate on visual inputs alone from a camera to minimize the necessary cost and hardware footprint. A compute-in-memory approach will be employed to minimize the power dissipation of the platform. Specifically, the research will investigate the co-designing of navigational models with physical and operating constraints of compute-in-memory to dramatically improve the platform's computational efficiency. To improve the robustness of prediction, deep learning framework of the low-power chip will also be augmented with a probabilistic inference. Using the procedure, not only the prediction itself, but the prediction confidence will also be extracted. Thereby, a drone will be made self-aware of when the mispredictions from deep learning models are likely due to dramatic changes in the flying scene. To operate under uncertainties, the drone will also encompass a computing framework based on probabilistic reasoning. The probabilistic framework will operate by considering many predictive hypotheses and sequentially filtering out the unlikely ones based on measurements. Unlike deep learning-based predictions which are extracted through a single-shot processing flow, predictions from reasoning are more energy expensive by considering a multitude of hypotheses and measurements. Therefore, deep learning and reasoning-based frameworks are also synergistically integrated to concurrently optimize robustness and energy efficiency. Processing cores in the developed platform will be reconfigurable for both deep learning and reasoning models to minimize the necessary resources.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

基于机器学习的自主导航为相机等电子传感器主动探索其应用空间提供了独特的机会。例如，自动飞行相机可以在农田中定位受感染的植物，以防止疾病传播。同样，在工业工厂中，自飞行气体传感器可以快速识别气体泄漏。因此，大多数电子传感器通过基于机器学习的自我导航而变得活跃，可以大大提高用例。然而，出于实用性，飞行器（即，无人机）必须足够小以不显眼并且不干扰用户，诸如办公室中的人。无人机的小尺寸对于在狭窄的空间中导航也是必要的。由于微型无人机只能携带微型电池的有效载荷，因此最大限度地减少基于机器学习的导航机载处理的功耗是非常关键的。此外，飞行空间可以是高度动态的，例如，在室内应用中将存在人的移动和照明条件的变化。因此，无人机的导航必须对这些因素具有弹性。这项研究预计将开发新的硬件实现，用于基于机器学习的自主导航，可以在微型电池上维持。新硬件还将具有最小的占地面积，便于与微型无人机集成。该平台还将稳健地处理现实世界的不确定性，例如室内照明条件和人员移动的变化。研究人员还将开展各种协同教育活动，例如在当地高中组织机器学习研讨会，开发机器学习硬件新课程，并通过这项研究指导本科生。 研究人员将专门开发一个平台，用于基于深度学习的无人机位置和方向的连续跟踪。该平台将仅使用摄像头的视觉输入，以最大限度地减少必要的成本和硬件占用空间。将采用内存计算方法来最大限度地降低平台的功耗。具体而言，该研究将研究导航模型的协同设计与内存计算的物理和操作限制，以显着提高平台的计算效率。为了提高预测的鲁棒性，低功耗芯片的深度学习框架也将增加概率推理。使用该过程，不仅预测本身，而且预测置信度也将被提取。因此，无人机将自我意识到深度学习模型的错误预测可能是由于飞行场景的巨大变化。为了在不确定性下运行，无人机还将包含一个基于概率推理的计算框架。概率框架将通过考虑许多预测假设并根据测量结果依次过滤掉不太可能的假设来运作。与通过单次处理流程提取的基于深度学习的预测不同，通过考虑大量假设和测量来进行推理的预测更耗费能源。因此，深度学习和基于推理的框架也被协同集成，以同时优化鲁棒性和能源效率。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

ENOS: Energy-Aware Network Operator Search in Deep Neural Networks

ENOS：深度神经网络中的能源感知网络运营商搜索

• DOI: 10.1109/access.2022.3192515
• 发表时间: 2022
• 期刊: IEEE Access
• 影响因子: 3.9
• 作者: Nasrin, Shamma;Shylendra, Ahish;Darabi, Nastaran;Tulabandhula, Theja;Gomes, Wilfred;Chakrabarty, Ankush;Trivedi, Amit Ranjan
• 通讯作者: Trivedi, Amit Ranjan

MF-Net: Compute-In-Memory SRAM for Multibit Precision Inference Using Memory-Immersed Data Conversion and Multiplication-Free Operators

• DOI: 10.1109/tcsi.2021.3064033
• 发表时间: 2021-01
• 期刊: IEEE Transactions on Circuits and Systems I: Regular Papers
• 作者: Shamma Nasrin;Diaa Badawi;A. Cetin;Wilfred Gomes;A. Trivedi

MC-CIM: Compute-in-Memory With Monte-Carlo Dropouts for Bayesian Edge Intelligence

MC-CIM：具有蒙特卡罗辍学的内存计算用于贝叶斯边缘智能

• DOI: 10.1109/tcsi.2022.3224703
• 发表时间: 2023
• 期刊: IEEE Transactions on Circuits and Systems I: Regular Papers
• 作者: Shukla, Priyesh;Nasrin, Shamma;Darabi, Nastaran;Gomes, Wilfred;Trivedi, Amit Ranjan
• 通讯作者: Trivedi, Amit Ranjan

Ultralow-Power Localization of Insect-Scale Drones: Interplay of Probabilistic Filtering and Compute-in-Memory

昆虫级无人机的超低功耗定位：概率过滤和内存计算的相互作用

• DOI: 10.1109/tvlsi.2021.3100252
• 发表时间: 2022
• 期刊: IEEE Transactions on Very Large Scale Integration (VLSI
• 作者: Shukla, Priyesh;Muralidhar, Ankith;Iliev, Nick;Tulabandhula, Theja;Fuller, Sawyer B.;Trivedi, Amit Ranjan
• 通讯作者: Trivedi, Amit Ranjan

Compute-in-Memory Upside Down: A Learning Operator Co-Design Perspective for Scalability

内存计算颠倒：学习算子协同设计可扩展性的视角

• DOI: 10.23919/date51398.2021.9474119
• 发表时间: 2021
• 期刊: Automation & Test in Europe Conference & Exhibition (DATE
• 作者: Nasrin, Shamma;Shukla, Priyesh;Jaisimha, Shruthi;Trivedi, Amit Ranjan
• 通讯作者: Trivedi, Amit Ranjan