Learning sensorimotor control with neuromorphic sensors: Toward hyperdimensional active perception
Learning sensorimotor control with neuromorphic sensors: Toward hyperdimensional active perception
复制标题
DOI：
10.1126/scirobotics.aaw6736
复制
发表时间：
2019-05-15
期刊：
影响因子：
25
通讯作者：
Aloimonos, Y.
中科院分区：
文献类型：
--
作者：
Mitrokhin, A.;Sutor, P.;Aloimonos, Y.
现代机器人技术的标志是能够将平台的感知与摩托车能力直接融合的能力 - 概念通常称为“主动感知”。然而，我们发现行动和感知通常被保存在分开的空间中，这是基于框架的传统视觉的结果，并且只有在当下和运动中才是连续的实体。这座桥由动态视觉传感器（DVS）越过，这是一种可以看到运动的神经形态摄像机。我们提出了一种将动作和感知共同编码为有意义，语义知情和通过使用高维二进制向量（HBV5）一致的单个空间的方法。我们将DVS用于视觉感知，并表明视觉组件可以与系统速度结合，以实现动态世界感知，这为实时导航和避免障碍物提供了机会。代理执行的动作直接与形成自己的“内存”所经历的感知绑定。此外，由于HBV5可以编码整个动作和感知的历史，从原子序列到任意序列 - 作为恒定量的矢量，因此自动求解记忆与深度学习的控制范式相结合。我们在四轮无人机自我移动推理任务和MVSEC（MultiveHicle立体声事件摄像机）数据集上演示了这些属性。
The hallmark of modern robotics is the ability to directly fuse the platform's perception with its motoric ability-the concept often referred to as "active perception." Nevertheless, we find that action and perception are often kept in separated spaces, which is a consequence of traditional vision being frame based and only existing in the moment and motion being a continuous entity. This bridge is crossed by the dynamic vision sensor (DVS), a neuromorphic camera that can see the motion. We propose a method of encoding actions and perceptions together into a single space that is meaningful, semantically informed, and consistent by using hyperdimensional binary vectors (HBV5). We used DVS for visual perception and showed that the visual component can be bound with the system velocity to enable dynamic world perception, which creates an opportunity for real-time navigation and obstacle avoidance. Actions performed by an agent are directly bound to the perceptions experienced to form its own "memory." Furthermore, because HBV5 can encode entire histories of actions and perceptions-from atomic to arbitrary sequences-as constant-sized vectors, autoassociative memory was combined with deep learning paradigms for controls. We demonstrate these properties on a quadcopter drone ego-motion inference task and the MVSEC (multivehicle stereo event camera) dataset.