RUI: CCSS: Collaborative Research: Cooperative Unmanned Aerial Vehicles Enabled Scalable Mobile Panoramic Video Surveillance

该项目研究了一种网络物理系统(CPS)，其中使用协作无人机(UAV)来实现可扩展的移动全景视频监控。该系统通过飞行一队协作无人机来生成移动实时视频全景图。这是一项综合了计算机科学和航空航天工程专业知识的跨学科协作努力。该项目展示了传感、控制和通信的融合。该项目将产生包括分布式视频拼接算法、可扩展分布式系统和协作最优飞行控制算法在内的成果。该方案显著改善了视频全景图结果的延时和误差等性能。它推动了视频处理和控制理论的前沿。这一跨学科的努力将促进和促进计算机科学和航空航天工程这两个研究界之间的横向合作。该项目旨在通过让代表人数不足的学生参与拟议的研究活动和改进课程设置来改善他们的教育。本课题的研究成果在遥感、环境采样与监测、国土安全等领域具有广泛的应用前景，本课题有三个研究方向：1)分布式层次化视频拼接，生成高可伸缩性的协作无人机移动视频全景图；2)无人机编队飞行综合最优协同控制，支持分布式视频拼接；3)集成、接口和通信，将分布式层次化视频拼接与编队飞行协同控制相结合。将开发一个原型系统进行评估。它有四个突出的优点。首先，分布式分层体系结构可扩展到移动摄像机，并具有容错能力。其次，它利用了视频帧的时间和空间特征来进行高效的计算。第三，综合最优协同飞行控制在一个统一的框架内处理多个协同目标，并提供计算高效、分布式和控制。第四，分布式视频拼接和无人机协同控制的统一，方便了移动视频全景图的生成。特别是，分布式视频拼接将拼接所需的密集计算的压力降至最低。它通过将拼接推到靠近摄像机的前面来保持视频帧之间的高度同步。本文提出的分层视频拼接算法对分布式系统理论和算法有重要贡献。此外，拼接算法利用了无人机视频之间的空间和时间相关性，为计算机图形学/视觉做出了新的贡献。所提出的最优协同控制方法将大大促进多车辆或多智能体系统的协同控制。它将许多具有挑战性的协作问题集成到一个统一的优化框架中。这种方法提供了许多所需的能力：封闭形式、分布式和基于局部信息的控制律、同步编队、协作跟踪和障碍物/碰撞避免。这些综合特征不仅使精确、实时的协同监视成为可能，而且将协同控制理论推向了一个新的地平线，适用于范围广泛的协同任务。

This project investigates a cyber-physical system (CPS) in which cooperative unmanned aerial vehicles (UAVs) are used to enable scalable mobile panoramic video surveillance. This system generates mobile real-time video panorama by flying a fleet of cooperative UAVs. It is an interdisciplinary collaborative effort that synthesizes the expertise of Computer Science and Aerospace Engineering. This project demonstrates a convergence of sensing, control and communication. The project will generate outcomes including distributed video stitching algorithms, scalable distributed systems, and cooperative optimal flight control algorithms. This project significantly improves the performance such as delay and error in video panorama results. It advances the frontiers of both video processing and control theory. This interdisciplinary effort will promote and contribute to crosscutting collaborations between two research communities: computer science and aerospace engineering. The project aims to improve the education of underrepresented student populations by involving them in the proposed research activities and enhancing course curriculum. Results of this project have broad applications to areas such as remote sensing, environmental sampling and monitoring, homeland security, etc. The project has three research thrusts: 1) distributed hierarchical video stitching to generate mobile video panorama across cooperative UAVs with high scalability, 2) integrated optimal cooperative control of UAV formation flying to support the distributed video stitching, and 3) integration, interfacing and communication to unite the distributed hierarchical video stitching and the cooperative control of formation flying. A prototype system will be developed for evaluation. It has four salient merits. First, the distributed hierarchical architecture is scalable to mobile cameras and fault-tolerant. Second, it exploits the temporal and spatial features of video frames for efficient computation. Third, the integrated optimal cooperative flight control addresses the multiple cooperative objectives in one unified framework, and provides a computationally efficient, distributed and control. Forth, the unification of distributed video stitching and cooperative control of UAV facilitates the generation of mobile video panorama. In particular, the distributed video stitching minimizes the stress of intensive computation required in stitching. It maintains high synchronization among video frames by pushing the stitching to the front close to the cameras. The proposed hierarchical video stitching significantly contributes to distributed system theory and algorithms. In addition, the stitching algorithms exploit the spatial and temporal correlation among UAV videos, yielding a novel contribution to computer graphics/vision. The proposed optimal cooperative control method will significantly advance the cooperative control of multi-vehicle or multi-agent systems. It integrates many challenging cooperative problems into one unified optimization framework. This method enables a number of desired capabilities: closed-form, distributed and local information based control law, synchronous formation, cooperative tracking, and obstacle/collision avoidance. These integrated features not only make precise and real-time cooperative surveillance possible, but also move forward the cooperative control theory to a new horizon for a wide range of cooperative missions.