Deep Optical Learning Devices and Architectures

在本提案中，作者引入了光学器件的新概念，光学整流线性单元（ReLU），并将其作为构建光学深度学习系统所需的关键使能技术。ReLU神经元的引入缓解了深度多层神经网络中梯度消失的问题，使最先进的深度神经网络的训练成为可能。作者提出了一种有效的光学实现ReLU，其中双向传输光开关由前向传播场的干涉差分检测器部分控制，既实现了线性前向响应的整流，又实现了深度学习所需的反向传播误差门控所需的导数。光学ReLU是实现可物理实现的可训练光学机器学习技术的关键组件。这种光学深度学习系统有潜力将性能扩展到一个水平，甚至远远超过对大规模并行超级计算机进一步发展的最乐观的预测，并且通过利用相干光子的高效模拟计算能力，将使用更少的能量。研究团队将使用硅上液晶智能像素技术设计、制造、测试和演示这些新型光学ReLU设备的大型阵列。然后将开发自对准深度学习光学系统的概念验证实验室演示。本项目将培养一名机器学习、深度神经网络、CMOS器件设计与制造、液晶化学与物理、相干与非线性光学、激光、计算机控制实验技术等领域的研究生。这样的跨学科背景将产生一个灵活的研究领导者，能够推进光学和机器学习科学和技术的前沿。在这个项目中，PI将继续开发傅里叶光学和全息领域的大规模开放在线课程（MOOC），这将有助于培养新一代光学科学家。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In this proposal the authors introduce a new concept for an optical device, the optical Rectifying Linear Unit or ReLU, and motivate it as the key enabling technology required for constructing an optical deep learning system. The ReLU neuron was introduced to alleviate the vanishing gradient problem in deep multi-layer neural networks, and it has allowed the training of state-of-the-art deep neural networks.The authors present an efficient optical implementation of the ReLU in which a bidirectional transmissive optical switch that is controlled by an interferometric differential detector part of the forward propagating field implements both the rectifying linear forward response but also the derivative needed for gating the backwards propagating error needed for deep learning. The optical ReLU is the key component necessary to realize the vision of a physically implemented trainable optical machine learning technology.Such an optical deep learning system has the potential to scale up in performance to a level far in excess even the most optimistic projections for the further development of massively parallel super computers and will use much less energy by harnessing the efficient analog computational capabilities of coherent photons. The research team will design, fabricate, test, and demonstrate large arrays of these new optical ReLU devices using liquid-crystal-on-Silicon smart-pixel technology. A proof-of-concept laboratory demonstration of a self-aligning deep learning optical system will then be developed.This project will train a graduate student in the fields of machine learning and deep neural networks as well as CMOS device design and fabrication, liquid crystal chemistry and physics, coherent and nonlinear optics, lasers, and computer controlled experimental technology. Such a cross-disciplinary background will produce a nimble research leader capable of advancing the frontiers of both optical and machine learning science and technology. During this program the PI will continue to develop a Massively Open Online Course (MOOC) in the areas of Fourier optics and holography that will help to train a new generation of optical scientists.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.