Architectures for Non-Image-Generating L2S Vision Systems

非图像生成 L2S 视觉系统的架构

• 批准号: 498557781
• 负责人: Professor Dr. Volker Blanz
• 金额: --
• 依托单位: Lehrstuhl Medieninformatik
• 依托单位国家: 德国
• 项目类别: Research Units
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/498557781?language=en
• 关键词: Architectures; Non; Image; Generating; L2S

The main goal of the Learning-to-Sense (L2S) research unit is to jointly optimize the design parameters of an application-specific sensor along with a neural network to analyze the resulting data.In this subproject, we explore and implement L2S architectures and machine learning (ML) algorithms for a number of novel sensor types that we develop together with partners in the research group. These sensors are for visible light and for mm-wave and THz synthetic aperture radar (SAR).The common feature of the proposed sensors is that they are (a) not designed for generating images, unlike cameras and many existing sensors, and (b) they use differential, contrast-based measurements. For visual light, the new sensors are partially motivated by findings in the human visual system, and they have a potential of being superior to standard cameras in high dynamic range environments. We propose to built sensors that have center-surround receptive fields and opponent-color coding (similar to retinal ganglion cells) and that calculate spatial and temporal derivatives on the analog signal on chip. In the research unit, we also develop and explore foveated sensors, and the focus of this subproject is on differential low-bandwidth data from the periphery of the visual field. For mm-waves and THz radiation, the proposed differential sensors remove substantial parts of the background signal which is due to direct reflections and multipath scattering, giving them the potential to simplify the inverse problems that are involved in computer vision.While many standard cameras and sensors can be optimized by improving on measures of image quality, the non-image-generating sensors can only be optimized and trained in an end-to-end application scenario with specific ML tasks. We explore such settings in the contexts of face and pedestrian detection, optical flow, peripheral vision and localization of structures in mm-wave and THz SAR. The results of this project contribute to the general development of the L2S paradigm, and provide several new and experimental approaches to computer vision.

学习感知（L2 S）研究单元的主要目标是联合优化特定应用传感器的设计参数，沿着神经网络来分析结果数据。在这个子项目中，我们与研究小组的合作伙伴共同开发了许多新型传感器类型，并探索和实现了L2 S架构和机器学习（ML）算法。这些传感器用于可见光以及毫米波和太赫兹合成孔径雷达（SAR）。拟议传感器的共同特点是：（a）与相机和许多现有传感器不同，它们不是为生成图像而设计的，（B）它们使用差分，基于对比度的测量。对于可见光，新传感器的部分动机是人类视觉系统的发现，它们在高动态范围环境中有可能上级标准相机。我们建议建立传感器，具有中心-周围感受野和彩色编码（类似于视网膜神经节细胞），并计算芯片上模拟信号的空间和时间导数。在研究单元中，我们还开发和探索中心凹传感器，这个子项目的重点是来自视野外围的差分低带宽数据。对于毫米波和太赫兹辐射，所提出的差分传感器消除了由于直接反射和多径散射而产生的大部分背景信号，从而有可能简化计算机视觉中涉及的逆问题。虽然许多标准相机和传感器可以通过改进图像质量来优化，非图像生成传感器只能在具有特定ML任务的端到端应用场景中进行优化和训练。我们探索这样的设置的背景下，人脸和行人检测，光流，周边视觉和定位的结构在毫米波和太赫兹SAR。该项目的结果有助于L2 S范式的总体发展，并为计算机视觉提供了几种新的实验方法。