CAREER:Bionic Eye: Heterogeneous Integration of Hemispherical Image Sensor with Artificial Neural Network

职业：仿生眼：半球图像传感器与人工神经网络的异构集成

• 批准号: 1942868
• 负责人: Kyusang Lee
• 金额: $ 50万
• 依托单位: University of Virginia Main Campus
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-02-01 至 2025-01-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1942868&HistoricalAwards=false
• 关键词: CAREER; Bionic; Eye; Heterogeneous; Integration

Non-Technical:Vision is one of the most crucial senses for humans when evaluating a scene. Natural imaging systems such as the eye provide an aberration-free image with a wide field of view. Artificial imaging systems can be made lighter, smaller, and with minimal optical aberrations by curving the focal plane array to match the curvature of the lens. Such a conformal architecture is analogous to the shape of the human eye. Furthermore, integration of an artificial synapse in image sensors will enable to processing of the high volumes of data acquired from a scene at low power. This approach is inspired by the interaction between the eye and visual cortex. In this project, the PI will design integrated hemispherical image sensors with an embedded neuromorphic chip that allows edge computing at hardware level with low-power operation. The system will provide wide-angle, aberration free images for various applications such as vehicle navigation, threat detection, and object identification. This will greatly facilitate further development of broad smart sensor applications combined with simplified AI technology. The PI will create introductory lab modules that connect basic material science and device physics to system level integration, allowing students to connect basic science with real-life applications.Technical:In this program, the PI will demonstrate an artificial retina integrated with neuromorphic circuits that consumes extremely low power. The PI will focus on designing and fabricating a thin-film InGaAs based hemispherical focal plane array and memrisror based artificial synapses that provides a wide field of view, simultaneous localization and mapping (SLAM), data reduction, ranging capability adaptable to variable and near infrared light levels while providing object recognition capabilities. This will be enabled by a combination of various unique technologies; from materials growth and lift-off, flexible device fabrication to heterogeneous and system level integration. The fabricated thin-film compound semiconductor based hemispherical image sensor array using remote epitaxy technique will be integrated with novel computing architecture based on emerging non-volatile resistive switching devices, providing an ideal implementation of a synaptic weight in artificial neural networks. This integrated image sensor will enable edge extraction by applying hexagonal kernel to the honeycomb image sensor array, and object recognition via a single layer of fully connected convolution neural network (CNN) similar to the human retina and visual cortex. Eventually, this proposed project will provide imagers that would enhance situational awareness and recognition with minimal power consumption in a power-constrained environment that is of central importance to robotics, autonomous driving, and military applications etc.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

非技术：视觉是人类在评估场景时最重要的感官之一。自然成像系统（例如眼睛）提供具有宽视场的无像差图像。通过弯曲焦平面阵列以匹配透镜的曲率，可以使人工成像系统更轻、更小并且具有最小的光学像差。这种共形架构类似于人眼的形状。此外，在图像传感器中集成人工突触将使得能够以低功率处理从场景获取的大量数据。这种方法的灵感来自于眼睛和视觉皮层之间的相互作用。在这个项目中，PI将设计集成半球形图像传感器，嵌入式神经形态芯片，允许低功耗操作的硬件级边缘计算。该系统将为车辆导航、威胁检测和物体识别等各种应用提供广角、无像差图像。这将极大地促进广泛的智能传感器应用与简化的人工智能技术相结合的进一步发展。PI将创建介绍性的实验模块，将基础材料科学和设备物理学与系统级集成联系起来，使学生能够将基础科学与实际应用联系起来。技术：在这个项目中，PI将展示一个集成了神经形态电路的人工视网膜，其功耗极低。PI将专注于设计和制造基于薄膜InGaAs的半球形焦平面阵列和基于记忆误差的人工突触，该人工突触提供宽视场、同时定位和映射（SLAM）、数据简化、适应可变和近红外光水平的测距能力，同时提供物体识别能力。这将通过各种独特技术的组合来实现;从材料生长和剥离，灵活的器件制造到异构和系统级集成。利用远程外延技术制造的基于薄膜化合物半导体的半球形图像传感器阵列将与基于新兴的非易失性电阻开关器件的新型计算架构集成，为人工神经网络中的突触权重提供理想的实现。这种集成图像传感器将通过将六边形内核应用于蜂窝图像传感器阵列来实现边缘提取，并通过类似于人类视网膜和视觉皮质的单层完全连接的卷积神经网络（CNN）来实现对象识别。最终，这个拟议的项目将提供成像器，将提高态势感知和识别，在功率受限的环境中，最小的功耗是至关重要的机器人，自动驾驶和军事应用等。这个奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

项目成果

Non-Line-of-Sight Detection Based on Neuromorphic Time-of-Flight Sensing

• DOI: 10.1021/acsphotonics.3c00448
• 发表时间: 2023-06
• 期刊: ACS Photonics
• 影响因子: 7
• 作者: Minseong Park;Yuan Yuan-Yuan;Y. Baek;B. Bae;Bo-In Park;Young Hoon Kim;Nicholas Lin;J. Heo;Kyusang Lee

2D materials-assisted heterogeneous integration of semiconductor membranes toward functional devices

• DOI: 10.1063/5.0122768
• 发表时间: 2022-11
• 期刊: Journal of Applied Physics
• 影响因子: 3.2
• 作者: Minseong Park;B. Bae;Taegeon Kim;H. Kum;Kyusang Lee

Quantized Neural Network via Synaptic Segregation Based on Ternary Charge‐Trap Transistors

• DOI: 10.1002/aelm.202300303
• 发表时间: 2023-09
• 期刊: Advanced Electronic Materials
• 影响因子: 6.2
• 作者: Y. Baek;B. Bae;Jeongyong Yang;Doeon Lee;H. Lee;Minseong Park;Taegeon Kim;Sihwan Kim;Bo-In Park;Geonwook Yoo;Kyusang Lee

Neuron‐Inspired Time‐of‐Flight Sensing via Spike‐Timing‐Dependent Plasticity of Artificial Synapses

• DOI: 10.1002/aisy.202100159
• 发表时间: 2021-11
• 期刊: Advanced Intelligent Systems
• 影响因子: 7.4
• 作者: Minseong Park;Yuan Yuan-Yuan;Y. Baek;A. Jones;Nicholas Lin;Doeon Lee;H. Lee;Sihwan Kim;J. Campbell;Kyusang Lee

Efficient Defect Identification via Oxide Memristive Crossbar Array Based Morphological Image Processing

• DOI: 10.1002/aisy.202000202
• 发表时间: 2021-02-01
• 期刊: ADVANCED INTELLIGENT SYSTEMS
• 影响因子: 7.4
• 作者: Lee, Hee Sung;Baek, Yongmin;Lee, Kyusang
• 通讯作者: Lee, Kyusang