Event-Based Vision: A Survey
Event-Based Vision: A Survey
复制标题
DOI：
10.1109/tpami.2020.3008413
复制
发表时间：
2022-01-01
影响因子：
23.6
通讯作者：
Scaramuzza, Davide
中科院分区：
文献类型：
--
作者：
Gallego, Guillermo;Delbruck, Tobi;Scaramuzza, Davide
事件相机是仿生传感器，与传统帧相机不同：它们不是以固定速率捕获图像，而是异步测量每个像素的亮度变化，并输出对亮度变化的时间、位置和符号进行编码的事件流。与传统相机相比，事件相机具有吸引人的特性：高时间分辨率（μ s 量级）、极高的动态范围（140 dB 与 60 dB）、低功耗和高像素带宽（kHz 量级），从而减少运动模糊。因此，事件相机在传统相机具有挑战性的场景中（例如低延迟、高速和高动态范围）在机器人和计算机视觉方面具有巨大潜力。然而，需要新的方法来处理这些传感器的非常规输出，以释放它们的潜力。本文全面概述了基于事件的视觉这一新兴领域，重点介绍了为释放事件相机的出色性能而开发的应用程序和算法。我们从事件相机的工作原理、可用的实际传感器以及它们所用于的任务来介绍事件相机，从低级视觉（特征检测和跟踪、光流等）到高级视觉（重建、分割、识别）。我们还讨论了为处理事件而开发的技术，包括基于学习的技术，以及用于这些新型传感器的专用处理器，例如尖峰神经网络。此外，我们还强调了在寻找一种更高效、受生物启发的方式让机器感知世界并与世界互动的过程中仍有待解决的挑战和机遇。
Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of mu s), very high dynamic range (140 dB versus 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision in challenging scenarios for traditional cameras, such as low-latency, high speed, and high dynamic range. However, novel methods are required to process the unconventional output of these sensors in order to unlock their potential. This paper provides a comprehensive overview of the emerging field of event-based vision, with a focus on the applications and the algorithms developed to unlock the outstanding properties of event cameras. We present event cameras from their working principle, the actual sensors that are available and the tasks that they have been used for, from low-level vision (feature detection and tracking, optic flow, etc.) to high-level vision (reconstruction, segmentation, recognition). We also discuss the techniques developed to process events, including learning-based techniques, as well as specialized processors for these novel sensors, such as spiking neural networks. Additionally, we highlight the challenges that remain to be tackled and the opportunities that lie ahead in the search for a more efficient, bio-inspired way for machines to perceive and interact with the world.