CAREER:Bionic Eye: Heterogeneous Integration of Hemispherical Image Sensor with Artificial Neural Network

非技术：视觉是人类评估场景时最重要的感官之一。像眼睛这样的自然成像系统提供无像差的宽视场图像。通过弯曲焦平面阵列以匹配透镜的曲率，可以使人工成像系统变得更轻、更小、光学像差最小。这种共形结构类似于人眼的形状。此外，在图像传感器中集成人工突触将能够以低功耗处理从场景中获取的大量数据。这种方法的灵感来自于眼睛和视觉皮层之间的相互作用。在该项目中，PI将设计带有嵌入式神经形态芯片的集成半球形图像传感器，以低功耗操作实现硬件级边缘计算。该系统将为各种应用提供广角、无像差的图像，如车辆导航、威胁检测和目标识别。这将极大地促进广泛的智能传感器应用与简化的人工智能技术的进一步发展。PI将创建入门实验室模块，将基础材料科学和设备物理与系统级集成联系起来，使学生能够将基础科学与现实应用联系起来。技术方面：在这个项目中，PI将展示一种集成了神经形态电路的人工视网膜，其功耗极低。该项目将专注于设计和制造基于薄膜InGaAs的半球形焦平面阵列和基于记忆镜的人工突触，提供宽视场、同步定位和映射（SLAM）、数据缩减、适应可变和近红外光水平的测距能力，同时提供目标识别能力。这将通过各种独特技术的结合来实现；从材料生长和起飞，柔性器件制造到异构和系统级集成。利用远程外延技术制备的基于薄膜化合物半导体的半球形图像传感器阵列将与基于新兴非易失性电阻开关器件的新型计算架构集成，为人工神经网络中的突触权提供了理想的实现。该集成图像传感器将通过在蜂窝图像传感器阵列上应用六边形核实现边缘提取，并通过类似于人类视网膜和视觉皮层的单层全连接卷积神经网络（CNN）实现物体识别。最终，这个拟议的项目将提供成像仪，以最小的功耗增强态势感知和识别，在电力受限的环境中，这对机器人、自动驾驶和军事应用等至关重要。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Non-Technical:Vision is one of the most crucial senses for humans when evaluating a scene. Natural imaging systems such as the eye provide an aberration-free image with a wide field of view. Artificial imaging systems can be made lighter, smaller, and with minimal optical aberrations by curving the focal plane array to match the curvature of the lens. Such a conformal architecture is analogous to the shape of the human eye. Furthermore, integration of an artificial synapse in image sensors will enable to processing of the high volumes of data acquired from a scene at low power. This approach is inspired by the interaction between the eye and visual cortex. In this project, the PI will design integrated hemispherical image sensors with an embedded neuromorphic chip that allows edge computing at hardware level with low-power operation. The system will provide wide-angle, aberration free images for various applications such as vehicle navigation, threat detection, and object identification. This will greatly facilitate further development of broad smart sensor applications combined with simplified AI technology. The PI will create introductory lab modules that connect basic material science and device physics to system level integration, allowing students to connect basic science with real-life applications.Technical:In this program, the PI will demonstrate an artificial retina integrated with neuromorphic circuits that consumes extremely low power. The PI will focus on designing and fabricating a thin-film InGaAs based hemispherical focal plane array and memrisror based artificial synapses that provides a wide field of view, simultaneous localization and mapping (SLAM), data reduction, ranging capability adaptable to variable and near infrared light levels while providing object recognition capabilities. This will be enabled by a combination of various unique technologies; from materials growth and lift-off, flexible device fabrication to heterogeneous and system level integration. The fabricated thin-film compound semiconductor based hemispherical image sensor array using remote epitaxy technique will be integrated with novel computing architecture based on emerging non-volatile resistive switching devices, providing an ideal implementation of a synaptic weight in artificial neural networks. This integrated image sensor will enable edge extraction by applying hexagonal kernel to the honeycomb image sensor array, and object recognition via a single layer of fully connected convolution neural network (CNN) similar to the human retina and visual cortex. Eventually, this proposed project will provide imagers that would enhance situational awareness and recognition with minimal power consumption in a power-constrained environment that is of central importance to robotics, autonomous driving, and military applications etc.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.