Reflexive robotics using asynchronous perception

该项目将开发一种完全不同的视觉感知方法&自主性，在这种方法中，图像本身的概念被一系列独立发射的像素取代，类似于视网膜中不同步的生物细胞。计算机视觉和机器学习的最新进展使机器人能够感知、理解并智能地与环境交互。然而，这种“解释性”行为只是自然界中发现的自主性的基本模式之一。该项目开发的技术将利用最近在瞬时、非基于图像的视觉传感方面的突破，使全新类型的自主系统成为可能。机器人能力的相应变化将影响制造业、太空、自动驾驶汽车和医疗部门。如果我们感知到一个物体高速接近，我们会本能地试图避开该物体，而不花时间解读场景。理解物体是什么或为什么它向我们靠近并不重要。这种“反身性”行为模式对于对时间紧迫的事件做出反应至关重要。在这种情况下，当我们有意识地意识到情况时，情况往往已经解决了。反身行为也是持续控制问题的重要组成部分。我们不愿在开车时将目光从道路上移开，因为我们知道，如果没有持续不断的感知和纠正循环，我们将迅速开始偏离航道。我们还发现，与完全依赖触觉感知相比，在观看对象的同时拾取和操纵对象要容易得多。不幸的是，视觉传感硬件需要巨大的带宽。百万像素相机每帧产生数百万字节。因此，时间采样率低，反应时间高，基于视觉数据的反射性调整变得不切实际。由于视觉传感器技术的最新进步，我们终于有机会颠覆视觉对于低延迟问题不切实际的范式，并促进机器人能力的阶段性变化。异步视觉传感器(也称为事件摄像机)避免定期的传感器范围更新(即图像)。取而代之的是，每个像素一旦检测到与其上一次传输的亮度变化，就独立且异步地传输一个信息包。这通过避免未改变的像素的冗余传输而显著地减少了数据带宽。更重要的是，因为这些包是立即传输的，传感器通常在事件发生和被感知之间提供3个数量级的延迟减少(30ms到30us)。视觉感知的这种进步是戏剧性的，但我们迫切需要机器人感知研究的相应革命。如果没有图像或同步采样的概念，几十年的计算机视觉和机器学习研究就无法使用这些传感器。该项目将通过开发新的感知和理解的异步范例，为机器人感知革命提供理论基础。这反映了生物系统，这将包括一个层次化的感知框架，包括低水平的反射和高水平的理解，其方式让人想起现代深度学习。然而，与深度学习不同的是，像素更新事件将异步发生，并将在系统中独立传播，从而保持极低的延迟。传感器技术仍处于早期试验阶段，很少有研究人员探索其对感知的影响。目前，国内或国际上没有任何团体在这一领域作出协调一致的努力。因此，该项目不仅为大量受生物启发的新的“反身机器人”应用奠定了基础。它还将支持一个独特的新研究团队的发展，使英国处于这一令人兴奋的领域的前沿。

This project will develop a fundamentally different approach to visual perception & autonomy where the concept of an image itself is replaced with a stream of independently firing pixels, similar to unsynchronised biological cells in the retina. Recent advances in computer vision & machine learning have enabled robots which can perceive, understand, and interact intelligently with, their environments. However, this "interpretive" behaviour is just one of the fundamental models of autonomy found in nature. The techniques developed in this project will exploit recent breakthroughs in instantaneous, non-image-based, visual sensing, to enable entirely new types of autonomous system. The corresponding step-change in robotic capabilities will impact the manufacturing, space, autonomous vehicles and medical sectors.If we perceive an object approaching at high speed, we instinctively try to avoid the object without taking the time to interpret the scene. It is not important to understand what the object is or why it's approaching us. This "reflexive" behavioural model is vital to react to time-critical events. In such cases, the situation has often already been resolved by the time we become consciously aware of it. Reflexive behaviour is also a vital component of continuous control problems. We are reluctant to take our eyes off the road while driving, as we know that we will rapidly begin to veer off course without a constant cycle of perception and correction. We also find it far easier to pick up and manipulate objects while looking at them, rather than relying entirely on tactile sensing. Unfortunately, visual sensing hardware requires enormous bandwidth. Megapixel cameras produce millions of bytes per frame. Thus, the temporal sampling rate is low, reaction times are high, and reflexive adjustments based on visual data become impractical.We finally have the opportunity to overturn the paradigm of vision being impractical for low-latency problems, and to facilitate a step change in robotic capabilities, thanks to recent advances in visual sensor technology. Asynchronous visual sensors (also known as event cameras) eschew regular sensor wide updates (i.e. images). Instead, every pixel independently and asynchronously transmits a packet of information, as soon as it detects an intensity change from its previous transmission. This drastically reduces data bandwidth by avoiding the redundant transmission of unchanged pixels. More importantly, because these packets are transmitted immediately, the sensor typically provides a latency reduction of 3 orders of magnitude (30ms to 30us) between an event occurring and it being perceived.This advancement in visual sensing is dramatic, but we are desperately in need of a commensurate revolution in robotic perception research. Without the concepts of the image or synchronous sampling, decades of computer vision and machine learning research is rendered unusable with these sensors. This project will provide the theoretical foundations for the robot perception revolution, by developing novel asynchronous paradigms for both perception and understanding. Mirroring biological systems, this will comprise a hierarchical perception framework encompassing both low-level reflexes and high-level understanding, in a manner reminiscent of modern deep-learning. However, unlike deep-learning, pixel-update events will occur asynchronously and will propagate independently through the system, hence maintaining extremely low latency.The sensor technology is still in its early trial phase, and few researchers are exploring its implications for perception. No group, nationally or internationally, is currently making a concerted effort in this area. Hence, this project not only lays the groundwork for a plethora of new biologically-inspired "reflexive robotics" applications. It will also support the development of a unique new research team, placing the UK at the forefront of this exciting field.