RUI: CCSS: Collaborative Research: Cooperative Unmanned Aerial Vehicles Enabled Scalable Mobile Panoramic Video Surveillance

RUI：CCSS：协作研究：协作无人机实现可扩展移动全景视频监控

• 批准号: 1407735
• 负责人: Ming Xin
• 金额: $ 17.82万
• 依托单位: University of Missouri-Columbia
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-09-01 至 2018-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1407735&HistoricalAwards=false
• 关键词: RUI; CCSS; Collaborative; Research; Cooperative

This project investigates a cyber-physical system (CPS) in which cooperative unmanned aerial vehicles (UAVs) are used to enable scalable mobile panoramic video surveillance. This system generates mobile real-time video panorama by flying a fleet of cooperative UAVs. It is an interdisciplinary collaborative effort that synthesizes the expertise of Computer Science and Aerospace Engineering. This project demonstrates a convergence of sensing, control and communication. The project will generate outcomes including distributed video stitching algorithms, scalable distributed systems, and cooperative optimal flight control algorithms. This project significantly improves the performance such as delay and error in video panorama results. It advances the frontiers of both video processing and control theory. This interdisciplinary effort will promote and contribute to crosscutting collaborations between two research communities: computer science and aerospace engineering. The project aims to improve the education of underrepresented student populations by involving them in the proposed research activities and enhancing course curriculum. Results of this project have broad applications to areas such as remote sensing, environmental sampling and monitoring, homeland security, etc. The project has three research thrusts: 1) distributed hierarchical video stitching to generate mobile video panorama across cooperative UAVs with high scalability, 2) integrated optimal cooperative control of UAV formation flying to support the distributed video stitching, and 3) integration, interfacing and communication to unite the distributed hierarchical video stitching and the cooperative control of formation flying. A prototype system will be developed for evaluation. It has four salient merits. First, the distributed hierarchical architecture is scalable to mobile cameras and fault-tolerant. Second, it exploits the temporal and spatial features of video frames for efficient computation. Third, the integrated optimal cooperative flight control addresses the multiple cooperative objectives in one unified framework, and provides a computationally efficient, distributed and control. Forth, the unification of distributed video stitching and cooperative control of UAV facilitates the generation of mobile video panorama. In particular, the distributed video stitching minimizes the stress of intensive computation required in stitching. It maintains high synchronization among video frames by pushing the stitching to the front close to the cameras. The proposed hierarchical video stitching significantly contributes to distributed system theory and algorithms. In addition, the stitching algorithms exploit the spatial and temporal correlation among UAV videos, yielding a novel contribution to computer graphics/vision. The proposed optimal cooperative control method will significantly advance the cooperative control of multi-vehicle or multi-agent systems. It integrates many challenging cooperative problems into one unified optimization framework. This method enables a number of desired capabilities: closed-form, distributed and local information based control law, synchronous formation, cooperative tracking, and obstacle/collision avoidance. These integrated features not only make precise and real-time cooperative surveillance possible, but also move forward the cooperative control theory to a new horizon for a wide range of cooperative missions.

该项目研究了一种网络物理系统（CPS），其中使用协作式无人机（uav）来实现可扩展的移动全景视频监控。该系统通过多架无人机协同飞行，生成移动实时全景视频。这是一个跨学科的合作努力，综合了计算机科学和航空航天工程的专业知识。该项目展示了传感、控制和通信的融合。该项目将产生包括分布式视频拼接算法、可扩展分布式系统和协作最优飞行控制算法在内的成果。该方案显著改善了视频全景结果的延迟和误差等性能。它推进了视频处理和控制理论的前沿。这种跨学科的努力将促进和促进两个研究团体之间的横切合作：计算机科学和航空航天工程。该项目旨在通过让代表性不足的学生参与拟议的研究活动和加强课程设置，改善对他们的教育。该项目的成果在遥感、环境采样与监测、国土安全等领域具有广泛的应用前景。本课题主要有三个研究方向：1)分布式分层视频拼接，生成具有高扩展性的跨协作无人机的移动视频全景图；2)集成无人机编队飞行最优协同控制，支持分布式视频拼接；3)集成、接口和通信，将分布式分层视频拼接与编队飞行协同控制统一起来。将开发一个原型系统进行评估。它有四个突出的优点。首先，分布式分层架构具有移动相机的可扩展性和容错性。其次，利用视频帧的时空特征进行高效的计算。第三，集成最优协同飞行控制在一个统一的框架内解决多个协同目标，提供了计算高效、分布式和控制的特性。四是将分布式视频拼接与无人机协同控制相统一，便于移动视频全景的生成。特别是分布式视频拼接，最大限度地减少了拼接过程中密集计算的压力。它通过将拼接推到靠近摄像头的前面来保持视频帧之间的高度同步。本文提出的分层视频拼接对分布式系统理论和算法有重要贡献。此外，拼接算法利用无人机视频之间的空间和时间相关性，为计算机图形学/视觉做出了新的贡献。所提出的最优协同控制方法对多车辆或多智能体系统的协同控制具有重要的促进作用。它将许多具有挑战性的协作问题集成到一个统一的优化框架中。该方法实现了许多期望的功能：封闭形式，分布式和基于本地信息的控制律，同步形成，协作跟踪和障碍物/碰撞避免。这些综合特点不仅使精确、实时的协同监视成为可能，而且将协同控制理论推向了一个新的高度，适用于广泛的协同任务。