Deep Optical Learning Devices and Architectures

在这项提案中，作者介绍了光学设备的一个新概念，光学整流线性单元或ReLU，并将其作为构建光学深度学习系统所需的关键技术。引入ReLU神经元是为了缓解深度多层神经网络中的梯度消失问题，它允许训练国家的作者提出了ReLU的有效光学实现，其中由前向传播场的干涉差分检测器部分控制的双向透射光开关实现了整流线性前向响应，也是门控深度学习所需的反向传播误差所需的导数。 光学ReLU是实现物理实现的可训练光学机器学习技术愿景所必需的关键组件，这样的光学深度学习系统有可能在性能上扩展到远远超过大规模并行超级计算机进一步发展的最乐观预测的水平，并且通过利用相干光子的高效模拟计算能力，将使用更少的能源。研究团队将使用硅基液晶智能像素技术设计、制造、测试和演示这些新型光学ReLU设备的大型阵列。该项目将培养一名研究生，学习机器学习和深度神经网络，以及CMOS器件设计和制造、液晶化学和物理、相干和非线性光学、激光和计算机控制实验技术。 这样的跨学科背景将产生一个灵活的研究领导者，能够推进光学和机器学习科学技术的前沿。在此期间，PI将继续开发傅立叶光学和全息领域的大规模开放在线课程（MOOC），以帮助培养新一代光学科学家。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响力审查标准进行评估，被认为值得支持。

In this proposal the authors introduce a new concept for an optical device, the optical Rectifying Linear Unit or ReLU, and motivate it as the key enabling technology required for constructing an optical deep learning system. The ReLU neuron was introduced to alleviate the vanishing gradient problem in deep multi-layer neural networks, and it has allowed the training of state-of-the-art deep neural networks.The authors present an efficient optical implementation of the ReLU in which a bidirectional transmissive optical switch that is controlled by an interferometric differential detector part of the forward propagating field implements both the rectifying linear forward response but also the derivative needed for gating the backwards propagating error needed for deep learning. The optical ReLU is the key component necessary to realize the vision of a physically implemented trainable optical machine learning technology.Such an optical deep learning system has the potential to scale up in performance to a level far in excess even the most optimistic projections for the further development of massively parallel super computers and will use much less energy by harnessing the efficient analog computational capabilities of coherent photons. The research team will design, fabricate, test, and demonstrate large arrays of these new optical ReLU devices using liquid-crystal-on-Silicon smart-pixel technology. A proof-of-concept laboratory demonstration of a self-aligning deep learning optical system will then be developed.This project will train a graduate student in the fields of machine learning and deep neural networks as well as CMOS device design and fabrication, liquid crystal chemistry and physics, coherent and nonlinear optics, lasers, and computer controlled experimental technology. Such a cross-disciplinary background will produce a nimble research leader capable of advancing the frontiers of both optical and machine learning science and technology. During this program the PI will continue to develop a Massively Open Online Course (MOOC) in the areas of Fourier optics and holography that will help to train a new generation of optical scientists.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.