Insect-inspired depth perception

任何想要与物体互动的动物或机器人都需要获得有关其3D形状的信息。人类使用立体视觉（两只眼睛的两个视图）来获得有关深度的信息，但需要大的大脑来处理这些信息。机器人也被建造成使用立体视觉，或其他类型的深度传感器，使用投射光或反射光。但这些都有一些限制，如能源消耗，对照明条件的敏感性，以及所需的计算处理量。我们感兴趣的是昆虫如何解决3D传感问题，具有小复眼和小大脑（总共约100，000个神经元），以及这是否为机器人提供了替代解决方案。昆虫，如果蝇（Drosophila），可以用高速/高分辨率的神经活动和行为记录进行研究。这表明它们使用一种特殊的机制来获取深度信息，这涉及到眼睛中单个光受体的运动。眼睛（与传统相机不同）记录相对的光线变化。在果蝇中，单个光敏细胞-对应于场景的单个“像素”-通过产生快速的反向运动来对这些光变化做出反应，我们称之为感光器微扫视。每个感光器在复眼内的特定位置以特定的方向移动，短暂地重新调整自己的光输入。感光器微扫视在左眼和右眼中是镜像对称的，这意味着相同的光线变化使它们同时向相反的方向移动。因此，在双眼观察时，一只眼睛的像素会随着世界而移动，而另一只眼睛的像素则会与之相反。最终，这些相反的微扫视会在眼睛和大脑网络的电信号中产生微小的时间差异，从而快速准确地告知苍蝇3D世界的结构。我们现在想确定果蝇大脑网络如何利用这种镜像对称的左眼和右眼信息来产生超分辨率立体视觉。我们将建立现实的模型，双目立体信息处理的苍蝇，并使用这些再现和预测的反应，三维物体。我们将测试这种编码的效率在人工神经网络（ANN）模拟驱动的微扫视采样。这种方法将与使用3D物体刺激监测神经活动的果蝇实验相结合，并使用行为测试来揭示动物的3D感知能力。我们的假设功能，然后将实现和测试的硬件，以确定是否可以获得相同的深度传感能力，使用传统的相机输入处理的一种新的方式，或通过设计一种新颖的光传感阵列，其中包括单独的移动的元素。其结果将是一种有效检测3D形状的新方法，该方法将具有多种潜在应用，例如用于机器人抓取任务。

Any animal, or robot, that wants to interact with objects needs to obtain information about their 3D shape. Humans use stereo vision (two views from two eyes) to gain information about depth, but require large brains to process this information. Robots have also been built that use stereo vision, or other kinds of depth sensors that use projected light or reflected light. But these have a number of limitations, such as energy consumption, sensitivity to lighting conditions, and the amount of computational processing needed. We are interested how insects solve the problem of 3D sensing, with small compound eyes and a tiny brain (altogether ~100,000 neurons), and whether this provides an alternative solution for robotics. Insects such as fruit flies (Drosophila) can be studied with high-speed/high-resolution neural activity and behaviour recordings. This has revealed they use a special mechanism to get depth information, which involves motion of the individual light receptors in the eye. Eyes (unlike conventional cameras) register relative light change. In Drosophila, individual light sensitive cells - corresponding to individual "pixels" of the scene - react to these light changes by generating an fast counter-motion, which we call a photoreceptor microsaccade. Each photoreceptor moves in a specific direction at its particular location inside the compound eye, transiently readjusting its own light input. The photoreceptor microsaccades are mirror-symmetric in the left and right eyes, meaning that the same light change makes them move simultaneously in opposite directions. Therefore, during binocular viewing, the pixels in one eye move transiently with the world and in the other eye against it. Ultimately, these opposing microsaccades should cause small timing differences in the eye and the brain networks' electrical signals, rapidly and accurately informing the fly of the 3D world structure. We now want to determine exactly how the Drosophila brain networks utilise this mirror-symmetric left and right eye information to produce super-resolution stereo vision. We will build realistic models of binocular stereo information processing in the fly and use these to reproduce and predict responses to 3D objects. We will test the efficiency of this encoding in Artificial Neural Network (ANN) simulations driven by microsaccadic sampling. This approach will be combined with experiments on Drosophila that monitor neural activity using 3D object stimulation, and use behavioural tests to reveal the animal's 3D perception capabilities. Our hypotheses about function will then be realised and tested in hardware, to determine if the same depth sensing capabilities can be obtained using either conventional camera input processed in a novel way, or through the design of a novel light sensing array that incorporates individual movement of the elements. The outcome will be a new method to efficiently detect 3D shape, which would have multiple potential applications, e.g. for robot grasping tasks.