Collaborative Research: SHF: Small: Quasi Weightless Neural Networks for Energy-Efficient Machine Learning on the Edge

合作研究：SHF：小型：用于边缘节能机器学习的准失重神经网络

• 批准号: 2326895
• 负责人: Subhasish Mitra
• 金额: $ 15万
• 依托单位: Stanford University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2326895&HistoricalAwards=false
• 关键词: Collaborative; Research; SHF; Small; Quasi

Deep Neural Networks (DNNs) have recently enabled revolutionary advances in a wide variety of tasks, however these deep networks demand large amounts of memory and computation resources. Such demands can be highly difficult (or even impractical) for systems on the edge. Although DNNs are very accurate, the energy consumed by DNNs is orders of magnitude higher than biological neural activities for similar tasks. It is important to reduce the computational and energy demands of machine learning hardware so that inferencing on the edge can become a low-cost, low-energy task. Weightless Neural Networks (WNNs) represent a distinct class of neural models which derive inspiration from the processing of input signals by the dendritic trees of biological neurons. WNNs do not use weights or multiply-add operations to determine their responses. Instead, they rely on value lookups implemented using look-up tables. This project explores small models that are more energy efficient compared to multiplication-and-addition-based deep learning models. WNNs are very promising from the perspective of energy-efficiency, and low latency, and our effort is directed at enabling a myriad of ultra-low energy edge applications otherwise impossible. This project explores low-energy machine learning hardware which combine the benefits of traditional DNNs and the computation-less weightless neural networks. Techniques used include (1) utilizing multi-layer networks and hierarchical networks to create novel weightless neural network architectures, (2) devising novel training algorithms for WNNs utilizing multi-shot training with feedback (3) exploring quasi-weightless neural networks using emerging novel memory technologies, and (4) designing systems for energy-efficient edge intelligence. The collaborative project between the University of Texas and Stanford University innovates across multiple layers of the system stack, including architecture and circuit layers. The collaborative activity between the University of Texas and Stanford involves many underrepresented communities from a STEM perspective, including minority and women, undergrads, and first-generation college students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度神经网络（DNN）最近在各种任务中取得了革命性的进展，但这些深度网络需要大量的内存和计算资源。对于边缘系统来说，这样的要求可能非常困难（甚至不切实际）。虽然DNN非常准确，但DNN消耗的能量比类似任务的生物神经活动高出几个数量级。 重要的是要减少机器学习硬件的计算和能源需求，以便边缘推理可以成为低成本，低能耗的任务。无权重神经网络（Weightless Neural Networks，WNN）是一类独特的神经模型，其灵感来自于生物神经元的树突树对输入信号的处理。 WNN不使用权重或乘加运算来确定其响应。相反，它们依赖于使用查找表实现的值查找。 该项目探索了与基于乘法和加法的深度学习模型相比更节能的小型模型。从能源效率和低延迟的角度来看，WNN非常有前途，我们的努力旨在实现无数的超低能耗边缘应用。该项目探索低能耗机器学习硬件，联合收割机了传统DNN和计算量较小的无重量神经网络的优点。 所使用的技术包括：（1）利用多层网络和分层网络来创建新的无重量神经网络架构;（2）利用带反馈的多次训练为WNN设计新的训练算法;（3）使用新兴的新型存储器技术探索准无重量神经网络;以及（4）设计节能边缘智能系统。德克萨斯大学和斯坦福大学之间的合作项目跨越系统堆栈的多个层进行创新，包括架构和电路层。德克萨斯大学和斯坦福大学之间的合作活动涉及许多从STEM角度来看代表性不足的社区，包括少数民族和妇女，本科生和第一代大学生。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估而被认为值得支持。