A Neuromorphic Proto-Object Based Dynamic Visual Saliency Model With a Hybrid FPGA Implementation.
A Neuromorphic Proto-Object Based Dynamic Visual Saliency Model With a Hybrid FPGA Implementation.
复制标题
DOI：
10.1109/tbcas.2021.3089622
复制
发表时间：
2021-06
影响因子：
5.1
通讯作者：
Etienne-Cummings R
中科院分区：
文献类型：
--
作者：
Molin J;Thakur C;Niebur E;Etienne-Cummings R
计算和关注视觉场景的显著区域是生物和工程系统执行高级视觉任务（包括对象检测、跟踪和分类）的固有和必要的预处理步骤。通过优先将计算资源投入到视野的显著区域来提高计算带宽和速度。人类大脑毫不费力地计算显着性，但在工程系统中建模这项任务具有挑战性。首先，我们提出了一个神经形态的动态显着性模型，这是自下而上，前馈，并基于与神经生理时空特征的原型对象的概念，不需要训练。我们的神经形态模型在预测人眼注视（即，地面真实显着性）。其次，我们提出了一个混合的FPGA实现的实时应用模型，能够处理112 × 84分辨率的帧在18.71 Hz，100 MHz的时钟速率运行-从软件实现的23.77倍加速。此外，我们的固定点模型的FPGA实现产生可比的结果的软件实现。
Computing and attending to salient regions of a visual scene is an innate and necessary preprocessing step for both biological and engineered systems performing high-level visual tasks including object detection, tracking, and classification. Computational bandwidth and speed are improved by preferentially devoting computational resources to salient regions of the visual field. The human brain computes saliency effortlessly, but modeling this task in engineered systems is challenging. We first present a neuromorphic dynamic saliency model, which is bottom-up, feed-forward, and based on the notion of proto-objects with neurophysiological spatio-temporal features requiring no training. Our neuromorphic model outperforms state-of-the-art dynamic visual saliency models in predicting human eye fixations (i.e., ground truth saliency). Secondly, we present a hybrid FPGA implementation of the model for real-time applications, capable of processing 112 × 84 resolution frames at 18.71 Hz running at a 100 MHz clock rate — a 23.77× speedup from the software implementation. Additionally, our fixed-point model of the FPGA implementation yields comparable results to the software implementation.