Deep Weakly-Supervised Neural Networks for Cross-Domain Video Recognition and Localization

用于跨域视频识别和定位的深度弱监督神经网络

• 批准号: RGPIN-2022-05397
• 负责人: Granger, Eric
• 金额: $ 2.55万
• 依托单位: École de technologie supérieure
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=753929
• 关键词: Deep; Weakly; Supervised; Neural; Networks

The proliferation of inexpensive sensors are paving the way for innovative AI technologies and applications. For instance, data from distributed RGB, IR, LiDAR and depth sensors are often integrated in, e.g., mobile robotics, autonomous driving, and video surveillance applications, to enhance perception. In such applications, cost-effective systems are required to recognize individuals, objects, and their behaviours from the massive amounts of data captured over multiple different sensors. Beyond the computational complexity, video recognition across multiple different domains (sensor modalities and operational conditions) may be degraded in real-world scenarios, due to cross-domain shifts, background clutter, variations in illumination, occlusion, etc. Deep learning (DL) models based on convolutional neural networks provide state-of-the-art performance in many visual recognition applications, yet their performance can decline in real-world applications when training on noisy data with limited or no annotations, and in the presence of domain shift between source and target (operational) data. The main long-term objective of this research program is to investigate and develop DL models for accurate video recognition and localization across multiple diverse domains with limited supervision. Given the high cost of collection and annotation of target data for training, these models will rely on weakly-supervised learning (WSL) to sustain a high level of performance. The specific objective consists in developing deep metric learning (DML) models based on weakly-labeled videos according to two main axes two main axes - (1) cross-domain adaptation and recognition, and (2) spatio-temporal object localization for interpretation. In Axis 1, knowledge distillation and multi-head methods are proposed to train DML models for weakly-supervised domain adaptation across different source and target domains. In Axis 2, methods for full-resolution class activation mapping methods and vision transformers are proposed to train DML models for weakly-supervised object localization and interpretation. Each axis will mainly be examined from a perspective of video re-identification applications using multiple distributed RGB and/or IR cameras, where video sequences are associated across different domains. This program will address concrete research problems related to the engineering of AI, and develop innovative methods that contribute to the state-of-art in several fields, most notably in DL and computer vision. Although this program will mostly focus on video-based recognition, these models are relevant in a wide range of tasks and applications. Given the recent popularity of DL, and it numerous commercial applications, this program will offer excellent opportunities for collaboration with academics, partnering with Canadian companies, training HQP, producing high impact publications, and initiating projects funded through other granting mechanisms.

廉价传感器的普及为创新的人工智能技术和应用铺平了道路。例如，来自分布式RGB， IR， LiDAR和深度传感器的数据通常集成在移动机器人，自动驾驶和视频监控应用中，以增强感知。在此类应用中，需要具有成本效益的系统从多个不同传感器捕获的大量数据中识别个人、物体及其行为。除了计算复杂性之外，由于跨域偏移、背景杂波、光照变化、遮挡等原因，在现实场景中，跨多个不同域（传感器模式和操作条件）的视频识别可能会下降。基于卷积神经网络的深度学习（DL）模型在许多视觉识别应用中提供了最先进的性能，但在现实应用中，当在带有有限或没有注释的噪声数据上进行训练时，以及在源和目标（操作）数据之间存在域转移时，它们的性能可能会下降。本研究计划的主要长期目标是在有限的监督下，研究和开发跨多个不同领域的准确视频识别和定位的深度学习模型。考虑到收集和标注目标数据用于训练的高成本，这些模型将依赖于弱监督学习（WSL）来维持高水平的性能。具体目标是根据两个主轴(1)跨域适应和识别，以及(2)用于解释的时空对象定位，开发基于弱标记视频的深度度量学习（DML）模型。在轴1中，提出了知识蒸馏和多头方法来训练DML模型，以便跨不同的源域和目标域进行弱监督域自适应。在轴2中，提出了全分辨率类激活映射方法和视觉变换方法来训练DML模型进行弱监督对象定位和解释。每个轴将主要从使用多个分布式RGB和/或IR相机的视频重新识别应用的角度进行检查，其中视频序列跨不同域相关联。该项目将解决与人工智能工程相关的具体研究问题，并开发创新方法，为几个领域的最新技术做出贡献，尤其是在深度学习和计算机视觉领域。虽然这个项目将主要集中在基于视频的识别上，但这些模型在广泛的任务和应用中都是相关的。鉴于DL最近的流行及其众多的商业应用，该计划将为与学术界合作、与加拿大公司合作、培训HQP、制作高影响力出版物以及启动通过其他资助机制资助的项目提供极好的机会。