Hardware based spatio-temporal neural processing backend for imaging sensors: Towards a smart camera
Hardware based spatio-temporal neural processing backend for imaging sensors: Towards a smart camera
复制标题
基于硬件的图像传感器时空神经处理后端：迈向智能相机
DOI：
10.1117/12.2305137
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Dutta, Achyut K.
中科院分区：
文献类型：
--
作者：
Gu, Yunfei;Ganguly, Samiran;Stan, Mircea R.;Ghosh, Avik W.;Dhar, Nibir K.;Dutta, Achyut K.
在这项工作中，我们展示了如何利用循环神经网络架构和生物学启发的训练方法的最新进展，为认知成像传感器构建一个技术平台。我们展示了特定于成像传感器的学习和处理任务，包括纯粹通过超出传感器材料基本限制的神经过滤来增强灵敏度和信噪比（SNR），以及这些网络在对象检测，运动跟踪和预测中的应用的推理和时空模式识别能力。然后，我们展示了使用互补金属氧化物半导体（CMOS）和新兴材料技术构建的单元硬件单元的设计，用于智能相机的超紧凑和节能嵌入式神经处理器。
In this work we show how we can build a technology platform for cognitive imaging sensors using recent advances in recurrent neural network architectures and training methods inspired from biology. We demonstrate learning and processing tasks specific to imaging sensors, including enhancement of sensitivity and signal-to-noise ratio (SNR) purely through neural filtering beyond the fundamental limits sensor materials, and inferencing and spatio-temporal pattern recognition capabilities of these networks with applications in object detection, motion tracking and prediction. We then show designs of unit hardware cells built using complementary metal-oxide semiconductor (CMOS) and emerging materials technologies for ultra-compact and energy-efficient embedded neural processors for smart cameras.