ATD: The Foundations of Dynamic Drone-Based Threat Detection

ATD：基于无人机的动态威胁检测的基础

• 批准号: 1737744
• 负责人: Guillermo Sapiro
• 金额: $ 19.99万
• 依托单位: Duke University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1737744&HistoricalAwards=false
• 关键词: ATD; Foundations; Dynamic; Drone; Based

Drone-based threat detection enables unprecedented coverage and flexibility in understanding human dynamics, with applications to real-time identification of unusual events and forecast of future threats. With these new possibilities come unique challenges, from highly dynamic scene changes to the need for low-cost operation. This project focuses on the foundations of video analysis technology for such dynamic drone-based threat detection. The work ranges from mathematical foundations in the area of learning and modeling to applications such as people tracking and identification. In terms of data, the project includes collection and analysis of drone-based video data, sharing data and the developed code with the community at large. The project will not only contribute to the emerging area of drone-based threat analysis but will also provide fundamental building blocks for modern visual data exploitation. Components of this project will be incorporated in online image-processing classes. The work investigates fundamental problems motivated by drone-based video analysis, including orientation invariance, image hashing, multi-modality modeling, and progressive unsupervised self-learning. The project develops and exploits underlying mathematical foundations, such as subspace modeling and invariant filter design. All the work has efficiency as its goal; this being manifested from the development of memory and computationally efficient forest hashing to the development of oriented response networks with significantly reduced deep models for orientation invariance. To enable state-of-the-art performance, the project utilizes successful machine learning frameworks, including deep convolution neural networks, random forests, hashing, and latent-SVM. This is approached with fundamental enabling redesigns and developments in the areas of robust learning, invariant learning, unsupervised self-learning, and multimodal hashing. The contributions are critical for data-limited learning, cross-modality learning, and computationally/memory efficient systems. The project aims to develop and exploit underlying mathematical foundations, such as subspace modeling, invariant filter design and learning, robust geometry-based learning, and information-based code aggregation. The theoretical and computational contributions are expected to result in efficient implementations of threat detection for dynamic environments, drone videos being a particularly important example.

基于无人机的威胁检测在理解人类动态方面实现了前所未有的覆盖范围和灵活性，应用于实时识别异常事件和预测未来威胁。这些新的可能性带来了独特的挑战，从高度动态的场景变化到对低成本操作的需求。本项目主要研究基于无人机的动态威胁检测的视频分析技术基础。工作范围从学习和建模领域的数学基础到人员跟踪和识别等应用。在数据方面，该项目包括收集和分析基于无人机的视频数据，与整个社区共享数据和开发的代码。该项目不仅将为基于无人机的威胁分析这一新兴领域做出贡献，还将为现代视觉数据开发提供基本的构建模块。这个项目的组成部分将被纳入在线图像处理课程。这项工作研究了基于无人机的视频分析引发的基本问题，包括方向不变性、图像哈希、多模态建模和渐进式无监督自学习。该项目开发和利用潜在的数学基础，如子空间建模和不变滤波器设计。一切工作都以效率为目标；这体现在内存和计算效率高的森林散列的发展，以及定向响应网络的发展，这些网络具有显著减少的定向不变性深度模型。为了实现最先进的性能，该项目利用了成功的机器学习框架，包括深度卷积神经网络、随机森林、哈希和潜在支持向量机。这是通过在鲁棒学习、不变学习、无监督自学习和多模态哈希等领域进行基本的重新设计和开发来实现的。这些贡献对于数据有限的学习、跨模态学习和计算/内存高效系统至关重要。该项目旨在开发和利用潜在的数学基础，如子空间建模、不变滤波器设计和学习、基于几何的鲁棒学习和基于信息的代码聚合。理论和计算的贡献预计将导致对动态环境进行有效的威胁检测，无人机视频是一个特别重要的例子。

项目成果

Detecting Adversarial Samples Using Influence Functions and Nearest Neighbors

• DOI: 10.1109/cvpr42600.2020.01446
• 发表时间: 2019-09
• 期刊: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Gilad Cohen;G. Sapiro;R. Giryes

Stop Memorizing: A Data-Dependent Regularization Framework for Intrinsic Pattern Learning

停止记忆：用于内在模式学习的数据依赖正则化框架

• DOI: 10.1137/19m1236886
• 发表时间: 2019
• 期刊: SIAM Journal on Mathematics of Data Science
• 影响因子: 3.6
• 作者: Zhu, Wei;Qiu, Qiang;Wang, Bao;Lu, Jianfeng;Sapiro, Guillermo;Daubechies, Ingrid
• 通讯作者: Daubechies, Ingrid

Using text to teach image retrieval

使用文本教授图像检索

• DOI: 10.1109/cvprw53098.2021.00180
• 发表时间: 2021
• 期刊: CVPR 2021 Workshop
• 作者: H. Dong, Z. Wang

Learning to Learn with Variational Information Bottleneck for Domain Generalization

• DOI: 10.1007/978-3-030-58607-2_12
• 发表时间: 2020-07
• 期刊: ArXiv
• 作者: Yingjun Du;Jun Xu;Huan Xiong;Qiang Qiu;Xiantong Zhen;Cees G. M. Snoek;Ling Shao

Minimax Pareto Fairness: A Multi Objective Perspective

• 发表时间: 2020-07
• 期刊: Proceedings of machine learning research
• 作者: Natalia Martínez;Martín Bertrán;G. Sapiro