Adaptive-SpikeNet: Event-based Optical Flow Estimation using Spiking Neural Networks with Learnable Neuronal Dynamics
Adaptive-SpikeNet: Event-based Optical Flow Estimation using Spiking Neural Networks with Learnable Neuronal Dynamics
复制标题
Adaptive-SpikeNet：使用具有可学习神经元动力学的尖峰神经网络进行基于事件的光流估计
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Kaushik Roy
中科院分区：
文献类型：
--
作者：
A. Kosta;Kaushik Roy
基于事件的摄像机最近显示出在高速运动估计方面的巨大潜力，因为它们能够异步捕获时间丰富的信息。尖峰神经网络(SNN)通过其神经启发的事件驱动处理可以有效地处理此类异步数据，而神经元模型(如泄漏积分和火灾(LIF))可以跟踪输入中包含的典型定时信息。SNN通过在神经元存储器中保持动态状态来实现这一点，在随着时间的推移保留重要信息的同时忘记冗余数据。因此，我们假设，与类似大小的模拟神经网络(ANN)相比，SNN将允许在顺序回归任务中获得更好的性能。然而，由于较晚层的尖峰消失，深层SNN很难训练。为此，我们提出了一种具有可学习神经元动力学的自适应全峰电位框架来缓解峰电位消失问题。我们利用基于代理梯度的时间反向传播算法(BPTT)从头开始训练我们的深层SNN。我们在多车辆立体事件相机(MVSEC)数据集和DSEC-FLOW数据集上验证了我们的方法的光流估计任务。我们在这些数据集上的实验表明，与最先进的人工神经网络相比，∼的平均端点误差平均降低了13%。我们还探索了几个缩小的模型，并观察到我们的SNN模型的性能始终优于类似大小的ANN，提供10%-16%的∼AEE降低。这些结果证明了SNN对于较小模型的重要性以及它们在边缘的适用性。在效率方面，我们的SNN在网络参数(∼48.3x)和计算能量(∼10.2x)方面提供了大量节省，同时与最先进的ANN实施方案相比，∼降低了10%。
Event-based cameras have recently shown great potential for high-speed motion estimation owing to their ability to capture temporally rich information asynchronously. Spiking Neural Networks (SNNs), with their neuro-inspired event-driven processing can efficiently handle such asynchronous data, while neuron models such as the leaky-integrate and fire (LIF) can keep track of the quintessential timing information contained in the inputs. SNNs achieve this by maintaining a dynamic state in the neuron memory, retaining important information while forgetting redundant data over time. Thus, we posit that SNNs would allow for better performance on sequential regression tasks compared to similarly sized Analog Neural Networks (ANNs). However, deep SNNs are difficult to train due to vanishing spikes at later layers. To that effect, we propose an adaptive fully-spiking framework with learnable neuronal dynamics to alleviate the spike vanishing problem. We utilize surrogate gradient-based backpropagation through time (BPTT) to train our deep SNNs from scratch. We validate our approach for the task of optical flow estimation on the Multi-Vehicle Stereo Event-Camera (MVSEC) dataset and the DSEC-Flow dataset. Our experiments on these datasets show an average reduction of ∼ 13% in average endpoint error (AEE) compared to state-of-the-art ANNs. We also explore several down-scaled models and observe that our SNN models consistently outperform similarly sized ANNs offering ∼10%-16% lower AEE. These results demonstrate the importance of SNNs for smaller models and their suitability at the edge. In terms of efficiency, our SNNs offer substantial savings in network parameters (∼ 48.3 ×) and computational energy (∼ 10.2 ×) while attaining ∼ 10% lower EPE compared to the state-of-the-art ANN implementations.
影响因子：
4.3
作者：
Brosch T;Tschechne S;Neumann H
通讯作者：
Neumann H