Comparing biological and computer vision: an AI approach to visual guidance in Harris' hawks.

深度学习技术在微型飞行器(MAV)飞行控制器的设计中越来越受欢迎。它们为这些车辆配备了对环境的感知，并允许它们做出智能导航决策。尽管已经取得了令人印象深刻的发展，但这些进步仍然落后于鸟类的表现，特别是在机动性和在城市或杂乱环境中的整合方面。为了进一步扩展MAV的潜力，我们需要它们避开障碍物，并以计算高效和传感器高效的方法跟踪目标，而不限制车辆有限的有效载荷能力。事实证明，RGB相机是非常有用的导航传感器。鸟类，尤其是猛禽，在飞行中强烈依赖视觉来执行目标跟踪和避障任务。我们能否从他们那里学到新的基于视觉的导航策略来解决这些导航问题？为了回答这个问题，我们提出了一个基于训练的Harris鹰的运动捕捉实验的框架，以及一个完全考虑视觉输入的捕捉鸟策略的深度学习模型。这种方法有几个优点：第一，深度学习模型非常适合以端到端的方式捕捉这种行为的复杂动态(从原始的感觉输入到高级控制命令)；第二，通过建立一个模仿鸟类的网络，我们可以评估视觉信息(纹理、光流、对比度变化等)的哪种编码。对于成功地模仿鸟类的行为，它是最相关的；第三，它使我们能够很容易地与目前在MAV中用于避障和目标跟踪的其他计算机视觉解决方案和导航方法进行比较。最后，我们考虑了仿鸟策略在模拟MAV制导中的应用。拟议的项目涉及BBSRC的几个优先领域：动物行为、生物学数学工具的使用、STEM生物学方法、生物科学的系统方法、生物科学的技术开发和数据驱动生物学。

Deep learning techniques are becoming increasingly popular in the design of flight controllers of micro-aerial vehicles (MAVs). They equip these vehicles with awareness of their environment, and allow them to take intelligent navigational decisions. Though impressive developments have been achieved, these advances still lag behind birds' performance, especially in terms of manoeuvrability and integration in urban or cluttered environments. In order to further expand the potential of MAVs, we need them to avoid obstacles and follow goals with computationally efficient and sensor-efficient methods, that don't restrict the vehicle's limited payload capacity. RGB cameras have proven to be very informative sensors for navigation. Birds, and raptors in particular, strongly rely on vision for carrying out target tracking and obstacle avoidance tasks in flight. Can we learn from them new vision-based navigational strategies to solve these guidance problems? To answer this question, we propose a framework based on motion-capture experiments with trained Harris' hawks, and a deep learning model that captures the bird's strategy, considering exclusively visual input. This approach provides several advantages: first, deep learning models are well-suited to capture the complex dynamics of such behaviours in an end-to-end fashion (from raw sensory inputs to high-level control commands); second, by building a bird-mimicking network we can evaluate which encodings of the visual information (texture, optic flow, contrast changes, etc.) are the most relevant for a successful imitation of the bird behaviour; third, it allows us to easily compare with other computer vision solutions and guidance approaches currently used in MAVs for obstacle avoidance and target tracking. Finally, we consider an application of the bird-imitating strategy to MAV guidance in simulation.The proposed project addresses several of the BBSRC priority areas: animal behaviour, use of mathematical tools for biology, STEM approaches to biology, systems approaches to the biosciences, technology development for the biosciences and data driven biology.