Insect-inspired depth perception

任何想要与物体互动的动物或机器人都需要获得有关物体3D形状的信息。人类使用立体视觉（两只眼睛的两个视角）来获取深度信息，但需要更大的大脑来处理这些信息。机器人也可以使用立体视觉，或者其他使用投影光或反射光的深度传感器。但这些都有一些局限性，比如能耗、对光照条件的敏感性以及所需的计算处理量。我们感兴趣的是，昆虫如何用小小的复眼和微小的大脑（总共约10万个神经元）解决3D传感问题，以及这是否为机器人技术提供了另一种解决方案。像果蝇这样的昆虫可以通过高速/高分辨率的神经活动和行为记录来研究。这表明它们使用一种特殊的机制来获取深度信息，这涉及到眼睛中单个光感受器的运动。眼睛（与传统相机不同）记录相对的光线变化。在果蝇中，单个感光细胞——对应于场景中的单个“像素”——通过产生快速的反运动来对这些光线变化做出反应，我们称之为光感受器微跳。每个光感受器在复眼内的特定位置向特定方向移动，短暂地重新调整自己的光输入。光感受器微眼珠在左右眼是镜面对称的，这意味着同样的光变化会使它们同时朝相反的方向运动。因此，在双眼观看时，一只眼睛中的像素随着世界的变化而瞬间移动，而另一只眼睛中的像素则与世界相反。最终，这些相反的微跳会在眼睛和大脑网络的电信号中产生微小的时间差，从而迅速准确地向苍蝇通报三维世界的结构。我们现在想要确定果蝇的大脑网络是如何利用这种镜像对称的左右眼信息来产生超分辨率立体视觉的。我们将在苍蝇中建立双目立体信息处理的现实模型，并使用这些模型来重现和预测对3D物体的反应。我们将在微进样驱动的人工神经网络（ANN）仿真中测试这种编码的效率。这种方法将与果蝇实验相结合，使用3D物体刺激来监测神经活动，并使用行为测试来揭示动物的3D感知能力。然后，我们对功能的假设将在硬件中实现和测试，以确定是否可以使用以新颖方式处理的传统相机输入或通过设计包含单个元素运动的新型光感测阵列来获得相同的深度感测能力。结果将是一种有效检测3D形状的新方法，这将有多种潜在的应用，例如机器人抓取任务。

Any animal, or robot, that wants to interact with objects needs to obtain information about their 3D shape. Humans use stereo vision (two views from two eyes) to gain information about depth, but require large brains to process this information. Robots have also been built that use stereo vision, or other kinds of depth sensors that use projected light or reflected light. But these have a number of limitations, such as energy consumption, sensitivity to lighting conditions, and the amount of computational processing needed. We are interested how insects solve the problem of 3D sensing, with small compound eyes and a tiny brain (altogether ~100,000 neurons), and whether this provides an alternative solution for robotics. Insects such as fruit flies (Drosophila) can be studied with high-speed/high-resolution neural activity and behaviour recordings. This has revealed they use a special mechanism to get depth information, which involves motion of the individual light receptors in the eye. Eyes (unlike conventional cameras) register relative light change. In Drosophila, individual light sensitive cells - corresponding to individual "pixels" of the scene - react to these light changes by generating an fast counter-motion, which we call a photoreceptor microsaccade. Each photoreceptor moves in a specific direction at its particular location inside the compound eye, transiently readjusting its own light input. The photoreceptor microsaccades are mirror-symmetric in the left and right eyes, meaning that the same light change makes them move simultaneously in opposite directions. Therefore, during binocular viewing, the pixels in one eye move transiently with the world and in the other eye against it. Ultimately, these opposing microsaccades should cause small timing differences in the eye and the brain networks' electrical signals, rapidly and accurately informing the fly of the 3D world structure. We now want to determine exactly how the Drosophila brain networks utilise this mirror-symmetric left and right eye information to produce super-resolution stereo vision. We will build realistic models of binocular stereo information processing in the fly and use these to reproduce and predict responses to 3D objects. We will test the efficiency of this encoding in Artificial Neural Network (ANN) simulations driven by microsaccadic sampling. This approach will be combined with experiments on Drosophila that monitor neural activity using 3D object stimulation, and use behavioural tests to reveal the animal's 3D perception capabilities. Our hypotheses about function will then be realised and tested in hardware, to determine if the same depth sensing capabilities can be obtained using either conventional camera input processed in a novel way, or through the design of a novel light sensing array that incorporates individual movement of the elements. The outcome will be a new method to efficiently detect 3D shape, which would have multiple potential applications, e.g. for robot grasping tasks.