RUI: CCSS: Collaborative Research: Cooperative Unmanned Aerial Vehicles Enabled Scalable Mobile Panoramic Video Surveillance

该项目研究了一个网络物理系统（CPS），其中合作的无人机（UAV）用于实现可扩展的移动的全景视频监控。该系统通过飞行一队合作的无人机来生成移动的实时视频全景。这是一个跨学科的合作努力，综合了计算机科学和航空航天工程的专业知识。该项目展示了传感、控制和通信的融合。该项目将产生包括分布式视频拼接算法、可扩展分布式系统和协作最优飞行控制算法在内的成果。该方案显著改善了视频全景图的延迟和误差等性能。它推进了视频处理和控制理论的前沿。这种跨学科的努力将促进和促进两个研究社区之间的交叉合作：计算机科学和航空航天工程。该项目旨在通过让代表性不足的学生参与拟议的研究活动和加强课程设置，改善他们的教育。该项目的成果在遥感、环境采样和监测、国土安全等领域有广泛的应用。1）分布式分层视频拼接以生成具有高可扩展性的跨协作UAV的移动的视频全景，2）UAV编队飞行的集成最优协作控制以支持分布式视频拼接，以及3）集成，接口和通信，以统一分布式分层视频拼接和编队飞行的协同控制。将开发一个原型系统进行评价。它有四个突出的优点。首先，分布式分层架构可扩展到移动的相机和容错。其次，它利用视频帧的时间和空间特征进行有效的计算。第三，综合最优协同飞行控制在一个统一的框架内解决了多个合作目标，并提供了一个计算效率高，分布式和控制。第四，分布式视频拼接与无人机协同控制的统一，有利于移动的视频全景图的生成。特别地，分布式视频拼接最小化了拼接中所需的密集计算的压力。它通过将拼接推到靠近摄像机的前面来保持视频帧之间的高度同步。提出的分层视频拼接算法对分布式系统理论和算法有重要的贡献。此外，拼接算法利用无人机视频之间的空间和时间的相关性，产生一个新的贡献，计算机图形学/视觉。所提出的最优协调控制方法将极大地推进多车辆或多智能体系统的协调控制。它集成了许多具有挑战性的合作问题到一个统一的优化框架。 该方法使一些所需的能力：封闭形式，分布式和本地信息为基础的控制律，同步形成，合作跟踪，和障碍物/碰撞避免。这些综合特性不仅使精确、实时的协同监视成为可能，而且将协同控制理论推向了一个新的领域，适用于各种协同任务。

This project investigates a cyber-physical system (CPS) in which cooperative unmanned aerial vehicles (UAVs) are used to enable scalable mobile panoramic video surveillance. This system generates mobile real-time video panorama by flying a fleet of cooperative UAVs. It is an interdisciplinary collaborative effort that synthesizes the expertise of Computer Science and Aerospace Engineering. This project demonstrates a convergence of sensing, control and communication. The project will generate outcomes including distributed video stitching algorithms, scalable distributed systems, and cooperative optimal flight control algorithms. This project significantly improves the performance such as delay and error in video panorama results. It advances the frontiers of both video processing and control theory. This interdisciplinary effort will promote and contribute to crosscutting collaborations between two research communities: computer science and aerospace engineering. The project aims to improve the education of underrepresented student populations by involving them in the proposed research activities and enhancing course curriculum. Results of this project have broad applications to areas such as remote sensing, environmental sampling and monitoring, homeland security, etc. The project has three research thrusts: 1) distributed hierarchical video stitching to generate mobile video panorama across cooperative UAVs with high scalability, 2) integrated optimal cooperative control of UAV formation flying to support the distributed video stitching, and 3) integration, interfacing and communication to unite the distributed hierarchical video stitching and the cooperative control of formation flying. A prototype system will be developed for evaluation. It has four salient merits. First, the distributed hierarchical architecture is scalable to mobile cameras and fault-tolerant. Second, it exploits the temporal and spatial features of video frames for efficient computation. Third, the integrated optimal cooperative flight control addresses the multiple cooperative objectives in one unified framework, and provides a computationally efficient, distributed and control. Forth, the unification of distributed video stitching and cooperative control of UAV facilitates the generation of mobile video panorama. In particular, the distributed video stitching minimizes the stress of intensive computation required in stitching. It maintains high synchronization among video frames by pushing the stitching to the front close to the cameras. The proposed hierarchical video stitching significantly contributes to distributed system theory and algorithms. In addition, the stitching algorithms exploit the spatial and temporal correlation among UAV videos, yielding a novel contribution to computer graphics/vision. The proposed optimal cooperative control method will significantly advance the cooperative control of multi-vehicle or multi-agent systems. It integrates many challenging cooperative problems into one unified optimization framework. This method enables a number of desired capabilities: closed-form, distributed and local information based control law, synchronous formation, cooperative tracking, and obstacle/collision avoidance. These integrated features not only make precise and real-time cooperative surveillance possible, but also move forward the cooperative control theory to a new horizon for a wide range of cooperative missions.