Collaborative Research: NRI: INT: Dense 3D Reconstruction of Dynamic Actors in Natural Environments using Multiple Flying Cameras

虽然已经开发了用于在受控实验室环境中进行数据收集的大规模多摄像头圆顶，但是在户外不可能实现类似水平的测量质量，而户外对这种数据收集有很大的潜在益处。例如，此类测量的使用包括奔跑的猎豹或人的身体动态，或者分析动物或鸟类的放牧行为。这导致科学家依赖于极其低效和危险的数据收集方法。例如，研究野生动物行为的生物学家试图预测动物的位置，并在特定位置放置一些只能提供有限数据的摄像机。本项目通过探索方法的研究和大规模数据收集工具的开发来应对这些挑战，该工具用于高分辨率和多视点的视觉记录和自然群体行为的运动分析（例如，动物群或人群）在非常大的环境（例如，沙漠平原或山边）使用一组飞行机器人。该项目开发的计算模型集成了计算机视觉和多智能体控制的基础知识，以测量3D中的演员组。通过该系统的开发，本项目将在感知和控制的交叉点上取得重大进展，包括：（1）对精确、快速和鲁棒的目标运动预测和相对状态估计方法的新研究，该方法快速估计机器人和演员的3D运动，具有很强的不确定性估计;（2）对感知感知的多目标多无人机安全运动规划问题进行了新的分解，允许基于一致的行动者预测不确定性模型和覆盖目标进行长期规划;（3）一种新的有保证的安全但自适应的反应式飞行控制范例，即使在大的干扰和飞行器动态变化的情况下也能够产生安全机动，并且能够利用先前的飞行经验进行实时自适应;（4）无人机动态场景三维重建的新理论，实现了多组角色的高分辨率网格和骨架重建。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

While large-scale multi-camera domes have been developed for data collection in controlled laboratory settings it is not possible to achieve a similar level of measurement quality outdoors where there is much potential benefit to such data collection. For example, use of such measurements include the body dynamics of a running cheetah, or people, or analyzing herding behaviors of animals or birds. This leads to scientists relying on extremely inefficient and dangerous data collection methods. For example, biologists studying the behaviors of wild animals try to predict where the animals will be and place some cameras which only give some limited data at specific locations. This project addresses such challenges by exploring the research of methods and development of a large-scale data collection tool for high-resolution and multi-viewpoint visual recording and motion analysis of natural group behaviors (e.g., herds of animals or groups of people) in-the-wild over very large environments (e.g., desert plains or mountain sides) using a team of flying robots. This project develops computational models that integrate the fundamentals of computer vision and multi-agent control to measure the group of actors in 3D. Through the development of this system, this project will make major advances in technology at the intersection of perception and control that include: (1) a new study of methods for precise, rapid, and robust target motion forecasting and relative state estimation that estimates the 3D motion of the robots and actors quickly with strong uncertainty estimates; (2) a new decomposition of the perception-aware multi-objective multi-UAV safe motion planning problem, that allows long-term planning based on consistent actor forecasting uncertainty models and coverage objectives; (3) a new guaranteed safe but adaptive paradigm for reactive flight control that is able to generate safety maneuvers even under large disturbances and vehicle dynamics changes, and that can leverage prior flight experience for real-time adaptation; (4) new theory of 3D reconstruction for dynamic scenes captured by UAVs that will enable high-resolution mesh and skeletal reconstruction of the groups of actors. The research outcome will be disseminated through multiple educational activities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.