Training Deep Photonic Convolutional Neural Networks With Sinusoidal Activations
Training Deep Photonic Convolutional Neural Networks With Sinusoidal Activations
复制标题
DOI：
10.1109/tetci.2019.2923001
复制
发表时间：
2021-06-01
影响因子：
5.3
通讯作者：
Tefas, Anastasios
中科院分区：
文献类型：
--
作者：
Passalis, Nikolaos;Mourgias-Alexandris, George;Tefas, Anastasios
深度学习 (DL) 在许多具有挑战性的问题上取得了最先进的性能。然而，深度学习需要强大的硬件来进行训练和部署，这增加了成本和能源需求，并使大规模应用变得尤其困难。认识到这些困难，人们提出了几种神经形态硬件解决方案，包括可以以接近光速处理信息的光子硬件，并且可以从光子系统上可用的巨大带宽中受益。然而，使用这些基于光子的神经形态架构的效果尚未得到充分理解和研究，这些架构施加了训练 DL 模型时通常不考虑的额外约束。本文的主要贡献是对训练可部署在采用正弦激活元件的光子硬件上的深度神经网络的可行性进行了广泛的研究，并开发了允许成功训练这些网络的方法，同时考虑到所使用硬件的物理限制。使用不同的深度学习架构和四个不同复杂度的数据集来广泛评估所提出的方法。
Deep learning (DL) has achieved state-of-the-art performance in many challenging problems. However, DL requires powerful hardware for both training and deployment, increasing the cost and energy requirements and rendering large-scale applications especially difficult. Recognizing these difficulties, several neuromorphic hardware solutions have been proposed, including photonic hardware that can process information close to the speed of light and can benefit from the enormous bandwidth available on photonic systems. However, the effect of using these photonic-based neuromorphic architectures, which impose additional constraints that are not usually considered when training DL models, is not yet fully understood and studied. The main contribution of this paper is an extensive study on the feasibility of training deep neural networks that can be deployed on photonic hardware that employ sinusoidal activation elements, along with the development of methods that allow for successfully training these networks, while taking into account the physical limitations of the employed hardware. Different DL architectures and four datasets of varying complexity were used for extensively evaluating the proposed method.