Higher order neural processing with input-adaptive dynamic weights on MoS2 memtransistor crossbars
Higher order neural processing with input-adaptive dynamic weights on MoS2 memtransistor crossbars
复制标题
DOI：
10.3389/femat.2022.950487
复制
发表时间：
2022-08
影响因子：
1.8
通讯作者：
Leila Rahimifard;Ahish Shylendra -Ahish-Shylendra -2180672644;Shamma Nasrin -Shamma-Nasrin -2180672195;Stephanie E. Liu -Stephanie-E.-Liu -2180672856;Vinod K. Sangwan -Vinod-K.-Sangwan -2180672356;Mark C. Hersam -Mark-C.-Hersam -2180672622;A. Trivedi
中科院分区：
文献类型：
--
作者：
Leila Rahimifard;Ahish Shylendra -Ahish-Shylendra -2180672644;Shamma Nasrin -Shamma-Nasrin -2180672195;Stephanie E. Liu -Stephanie-E.-Liu -2180672856;Vinod K. Sangwan -Vinod-K.-Sangwan -2180672356;Mark C. Hersam -Mark-C.-Hersam -2180672622;A. Trivedi
深度学习系统的复杂性不断增加，将传统计算技术推向了极限。虽然忆阻器是深度学习加速的主流技术之一，但它只适用于经典学习层，其中只有两个操作数（即权重和输入）同时处理。同时，为了提高新兴应用的深度学习的计算效率，需要并发处理许多操作数的各种非传统层正在变得流行。例如，超网络通过同时处理应用上下文的权重和输入来提高其预测鲁棒性。双电极忆阻器网格不能直接映射新兴层的高阶乘法神经相互作用。为了解决这一未满足的需求，我们提出了交叉处理使用双栅极记忆晶体管的基础上二维半导体MoS2。与忆阻器不同，忆阻晶体管的电阻状态可以被持久地编程并且可以由多个栅极电极主动地控制。因此，所讨论的记忆晶体管交叉开关实现了超出常规无源交叉开关的若干高级推断架构。例如，我们表明，潜行路径可以有效地抑制在忆晶体管交叉，而它们限制了无源忆阻器交叉的大小可扩展性。类似地，利用栅极端子来抑制交叉杆权重，动态地将AlexNet的完全连接层的忆体晶体管交叉杆中的偏置功率降低了20%。在超网络等新兴层上，在相同的crossbar单元中配置多个操作，在所考虑的网络情况下，将操作功率降低了1.15倍。
The increasing complexity of deep learning systems has pushed conventional computing technologies to their limits. While the memristor is one of the prevailing technologies for deep learning acceleration, it is only suited for classical learning layers where only two operands, namely weights and inputs, are processed simultaneously. Meanwhile, to improve the computational efficiency of deep learning for emerging applications, a variety of non-traditional layers requiring concurrent processing of many operands are becoming popular. For example, hypernetworks improve their predictive robustness by simultaneously processing weights and inputs against the application context. Two-electrode memristor grids cannot directly map emerging layers’ higher-order multiplicative neural interactions. Addressing this unmet need, we present crossbar processing using dual-gated memtransistors based on two-dimensional semiconductor MoS2. Unlike the memristor, the resistance states of memtransistors can be persistently programmed and can be actively controlled by multiple gate electrodes. Thus, the discussed memtransistor crossbar enables several advanced inference architectures beyond a conventional passive crossbar. For example, we show that sneak paths can be effectively suppressed in memtransistor crossbars, whereas they limit size scalability in a passive memristor crossbar. Similarly, exploiting gate terminals to suppress crossbar weights dynamically reduces biasing power by ∼20% in memtransistor crossbars for a fully connected layer of AlexNet. On emerging layers such as hypernetworks, collocating multiple operations within the same crossbar cells reduces operating power by ∼ 15 × on the considered network cases.