Event camera simulator design for modeling attention-based inference architectures
Event camera simulator design for modeling attention-based inference architectures
复制标题
DOI：
10.1007/s11554-021-01191-y
复制
发表时间：
2021-05
影响因子：
3
通讯作者：
Md Jubaer Hossain Pantho;Joel Mandebi Mbongue;Pankaj Bhowmik;C. Bobda
中科院分区：
文献类型：
--
作者：
Md Jubaer Hossain Pantho;Joel Mandebi Mbongue;Pankaj Bhowmik;C. Bobda
近年来，人们对实现在图像传感器的水平上集成越来越多的计算的方法越来越感兴趣。这一上升趋势已经引起了人们对开发新型事件相机的研究兴趣，这些相机可以直接在传感器中促进CNN计算。然而，基于事件的相机可能是昂贵的，限制了对高级模型和算法的性能探索。本文提出了一个事件摄像机模拟器，可以是一个有力的工具，硬件设计原型，参数优化，注意力为基础的创新算法开发，和基准测试。建议的模拟器实现了一个分布式计算模型，以识别图像帧中的相关区域。我们的模拟器的相关性计算模型实现为一个模块的集合，并执行并行计算。分布式计算模型是可配置的，这使得它对于设计空间探索非常有用。模拟器的渲染引擎仅在有新事件时对帧区域进行采样。模拟器紧密地模拟类似于物理相机的图像处理流水线。实验结果表明，该模拟器能够以较低的开销有效地模拟事件视觉
In recent years, there has been a growing interest in realizing methodologies to integrate more and more computation at the level of the image sensor. The rising trend has seen an increased research interest in developing novel event cameras that can facilitate CNN computation directly in the sensor. However, event-based cameras ca be expensive, limiting performance exploration on high-level models and algorithms. This paper presents an event camera simulator that can be a potent tool for hardware design prototyping, parameter optimization, attention-based innovative algorithm development, and benchmarking. The proposed simulator implements a distributed computation model to identify relevant regions in an image frame. Our simulator’s relevance computation model is realized as a collection of modules and performs computations in parallel. The distributed computation model is configurable, making it highly useful for design space exploration. The Rendering engine of the simulator samples frame-regions only when there is a new event. The simulator closely emulates an image processing pipeline similar to that of physical cameras. Our experimental results show that the simulator can effectively emulate event vision with low overheads