Addressing Unreliability in Emerging Devices and Non-von Neumann Architectures Using Coded Computing
Addressing Unreliability in Emerging Devices and Non-von Neumann Architectures Using Coded Computing
复制标题
DOI：
10.1109/jproc.2020.2986362
复制
发表时间：
2020-08-01
影响因子：
20.6
通讯作者：
Grover, Pulkit
中科院分区：
文献类型：
--
作者：
Dutta, Sanghamitra;Jeong, Haewon;Grover, Pulkit
计算系统正在迅速发展。在器件层面，新兴器件开始与传统CMOS系统竞争。在体系结构层面上，新的体系结构成功地避免了通信瓶颈，这是冯诺依曼体系结构的一个核心特征，也是一个核心限制。此外，这种系统越来越受到不可靠性的困扰。这种不可靠性出现在新兴设备的设备或门级，如果不加以检查，可能会渗透到处理器或系统级。本文的目的是调查最近的进展，在可靠的计算使用不可靠的元素，着眼于非硅和非冯诺依曼架构。我们首先观察到，社区可以使用“现代计算的侏儒”，而不是针对一般的计算问题，首先在高性能计算（HPC）社区中指出，作为起点。这些计算问题是当今几乎所有科学计算、机器学习和数据分析的基本构建块。接下来，我们调查了“编码计算”的最新技术水平，这是一个新兴的领域，它在经典的基于算法的容错（ABFT）上取得了进展，并带来了基本的信息理论观点。通过将纠错码编织到计算算法中，编码计算为已经开放了30多年的问题提供了显着的解决方案，并获得了新的基本限制。我们介绍现有的和新的编码计算技术的背景下，“编码的矮人”，其中一个特定的矮人的计算是弹性的应用编码。我们讨论了如何，对于相同的冗余，“编码侏儒”是更有弹性的经典技术，如复制。此外，通过研究一个广泛流行的计算任务训练大型神经网络，我们展示了如何编码的侏儒可以应用于解决这个根本上的非线性问题。最后，我们讨论了实际的挑战和未来的发展方向，在新兴和现有的非硅和/或非冯诺依曼架构实现编码计算技术。
Computing systems are evolving rapidly. At the device level, emerging devices are beginning to compete with traditional CMOS systems. At the architecture level, novel architectures are successfully avoiding the communication bottleneck that is a central feature, and a central limitation, of the von Neumann architecture. Furthermore, such systems are increasingly plagued by unreliability. This unreliability arises at device or gate-level in emerging devices, and can percolate up to processor or system-level if left unchecked. The goal of this article is to survey recent advances in reliable computing using unreliable elements, with an eye on nonsilicon and non-von Neumann architectures. We first observe that instead of aiming for generic computing problems, the community could use "dwarfs of modern computing," first noted in the high-performance computing (HPC) community, as a starting point. These computing problems are the basic building blocks of almost all scientific computing, machine learning, and data analytics today. Next, we survey the state of the art in "coded computing," which is an emerging area that advances on classical algorithm-based fault-tolerance (ABFT) and brings a fundamental information-theoretic perspective. By weaving error-correcting codes into a computing algorithm, coded computing provides dramatic improvements on solutions, as well as obtains novel fundamental limits, for problems that have been open for more than 30 years. We introduce existing and novel coded computing techniques in the context of "coded dwarfs," where a specific dwarf's computation is made resilient by applying coding. We discuss how, for the same redundancy, "coded dwarfs" are significantly more resilient compared to classical techniques such as replication. Furthermore, by examining a widely popular computation task-training large neural networks-we demonstrate how coded dwarfs can be applied to address this fundamentally nonlinear problem. Finally, we discuss practical challenges and future directions in implementing coded computing techniques on emerging and existing nonsilicon and/or non-von Neumann architectures.