HyDREA: Utilizing Hyperdimensional Computing For A More Robust and Efficient Machine Learning System
HyDREA: Utilizing Hyperdimensional Computing For A More Robust and Efficient Machine Learning System
复制标题
HyDREA：利用超维计算打造更强大、更高效的机器学习系统
DOI：
10.1145/3524067
复制
发表时间：
2022
影响因子：
2
通讯作者：
Rosing, Tajana
中科院分区：
文献类型：
--
作者：
Morris, Justin;Ergun, Kazim;Khaleghi, Behnam;Imani, Mohsen;Aksanli, Baris;Rosing, Tajana
今天的系统依赖于将所有数据发送到云端，然后使用复杂的算法，比如深度神经网络，这需要数十亿个参数和许多小时来训练一个模型。相比之下，人类的大脑可以毫不费力地完成大部分的学习。超高维（HD）计算旨在通过利用高维表示来模拟人类大脑的行为。这导致了其他机器学习（ML）算法所缺乏的各种理想特性，例如系统中对噪声的鲁棒性和简单，高度并行的操作。在本文中，我们提出了𝖧𝗒𝖣𝖱𝖤𝖠，一个鲁棒、高效和准确的超维计算系统。我们提出了一种内存中处理（PIM）架构，该架构适用于具有挑战性的通信场景的联邦学习环境，这些场景会导致传输数据中的错误。𝖧𝗒𝖣𝖱𝖤𝖠根据输入样本的信噪比（SNR）自适应改变模型的位宽，以保持高清模型的准确性，同时实现显著的加速和能效。在分类方面，我们的PIM架构能够实现比基线PIM架构提高28倍和255倍的能效；在集群方面，我们的PIM架构能够实现比基线PIM架构提高32倍的速度和289倍的能效。𝖧𝗒𝖣𝖱𝖤𝖠能够通过放松硬件参数来实现这一点，从而在引入计算错误的同时获得能源效率和加速。实验表明，由于其独特的鲁棒性，HD计算能够在不显着降低精度的情况下处理错误。对于无线噪声，我们发现𝖧𝗒𝖣𝖱𝖤𝖠对噪声的鲁棒性比其他可比较的ML算法强48倍。我们的结果表明，即使在信噪比为6.64的情况下，我们提出的系统也会损失不到1%的分类精度。我们还测试了在聚类应用中使用高清计算的鲁棒性，发现即使在信噪比低于7 dB的情况下，我们提出的系统在互信息得分上的损失也不到1%，这比K-means对噪声的鲁棒性高57倍。
Today’s systems rely on sending all the data to the cloud and then using complex algorithms, such as Deep Neural Networks, which require billions of parameters and many hours to train a model. In contrast, the human brain can do much of this learning effortlessly. Hyperdimensional (HD) Computing aims to mimic the behavior of the human brain by utilizing high-dimensional representations. This leads to various desirable properties that other Machine Learning (ML) algorithms lack, such as robustness to noise in the system and simple, highly parallel operations. In this article, we propose 𝖧𝗒𝖣𝖱𝖤𝖠, a HyperDimensional Computing system that is Robust, Efficient, and Accurate. We propose a Processing-in-Memory (PIM) architecture that works in a federated learning environment with challenging communication scenarios that cause errors in the transmitted data. 𝖧𝗒𝖣𝖱𝖤𝖠 adaptively changes the bitwidth of the model based on the signal-to-noise ratio (SNR) of the incoming sample to maintain the accuracy of the HD model while achieving significant speedup and energy efficiency. Our PIM architecture is able to achieve a speedup of 28× and 255× better energy efficiency compared to the baseline PIM architecture for Classification and achieves 32 × speed up and 289 × higher energy efficiency than the baseline architecture for Clustering. 𝖧𝗒𝖣𝖱𝖤𝖠 is able to achieve this by relaxing hardware parameters to gain energy efficiency and speedup while introducing computational errors. We show experimentally, HD Computing is able to handle the errors without a significant drop in accuracy due to its unique robustness property. For wireless noise, we found that 𝖧𝗒𝖣𝖱𝖤𝖠 is 48 × more robust to noise than other comparable ML algorithms. Our results indicate that our proposed system loses less than 1% Classification accuracy, even in scenarios with an SNR of 6.64. We additionally test the robustness of using HD Computing for Clustering applications and found that our proposed system also looses less than 1% in the mutual information score, even in scenarios with an SNR under 7 dB, which is 57 × more robust to noise than K-means.