Deep Weakly-Supervised Neural Networks for Cross-Domain Video Recognition and Localization

用于跨域视频识别和定位的深度弱监督神经网络

• 批准号: RGPIN-2022-05397
• 负责人: Granger, Eric
• 金额: $ 2.55万
• 依托单位: École de technologie supérieure
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=753929
• 关键词: Deep; Weakly; Supervised; Neural; Networks

The proliferation of inexpensive sensors are paving the way for innovative AI technologies and applications. For instance, data from distributed RGB, IR, LiDAR and depth sensors are often integrated in, e.g., mobile robotics, autonomous driving, and video surveillance applications, to enhance perception. In such applications, cost-effective systems are required to recognize individuals, objects, and their behaviours from the massive amounts of data captured over multiple different sensors. Beyond the computational complexity, video recognition across multiple different domains (sensor modalities and operational conditions) may be degraded in real-world scenarios, due to cross-domain shifts, background clutter, variations in illumination, occlusion, etc. Deep learning (DL) models based on convolutional neural networks provide state-of-the-art performance in many visual recognition applications, yet their performance can decline in real-world applications when training on noisy data with limited or no annotations, and in the presence of domain shift between source and target (operational) data. The main long-term objective of this research program is to investigate and develop DL models for accurate video recognition and localization across multiple diverse domains with limited supervision. Given the high cost of collection and annotation of target data for training, these models will rely on weakly-supervised learning (WSL) to sustain a high level of performance. The specific objective consists in developing deep metric learning (DML) models based on weakly-labeled videos according to two main axes two main axes - (1) cross-domain adaptation and recognition, and (2) spatio-temporal object localization for interpretation. In Axis 1, knowledge distillation and multi-head methods are proposed to train DML models for weakly-supervised domain adaptation across different source and target domains. In Axis 2, methods for full-resolution class activation mapping methods and vision transformers are proposed to train DML models for weakly-supervised object localization and interpretation. Each axis will mainly be examined from a perspective of video re-identification applications using multiple distributed RGB and/or IR cameras, where video sequences are associated across different domains. This program will address concrete research problems related to the engineering of AI, and develop innovative methods that contribute to the state-of-art in several fields, most notably in DL and computer vision. Although this program will mostly focus on video-based recognition, these models are relevant in a wide range of tasks and applications. Given the recent popularity of DL, and it numerous commercial applications, this program will offer excellent opportunities for collaboration with academics, partnering with Canadian companies, training HQP, producing high impact publications, and initiating projects funded through other granting mechanisms.

廉价传感器的扩散为创新的AI技术和应用铺平了道路。例如，分布式RGB，IR，LIDAR和DEPTH传感器的数据通常集成在例如移动机器人技术，自动驾驶和视频监视应用程序中，以增强感知。在这样的应用中，需要具有成本效益的系统来识别个人，对象及其行为，从多个不同传感器上捕获的大量数据。 Beyond the computational complexity, video recognition across multiple different domains (sensor modalities and operational conditions) may be degraded in real-world scenarios, due to cross-domain shifts, background clutter, variations in illumination, occlusion, etc. Deep learning (DL) models based on convolutional neural networks provide state-of-the-art performance in many visual recognition applications, yet their performance can decline in real-world applications when training on noisy data with有限或没有注释，并且在源和目标（操作）数据之间存在域移动的情况下。 该研究计划的主要长期目标是调查和开发DL模型，以在有限的监督下进行精确的视频识别和本地化。鉴于收集成本和培训目标数据的注释成本很高，这些模型将依靠弱监督的学习（WSL）来维持高水平的绩效。具体目标包括根据两个主轴开发基于弱标记的视频的深度度量学习（DML）模型，两个主轴 - （1）跨域的适应性和识别，以及（2）时空对象定位进行解释。在Axis 1中，提出了知识蒸馏和多头方法来训练DML模型，以跨不同源和目标域的弱监督域的适应性。在AXIS 2中，提出了全分辨率类激活映射方法和视觉变压器的方法来训练DML模型，以进行弱监督的对象定位和解释。每个轴将主要从视频重新识别应用程序的角度检查使用多个分布式RGB和/或IR摄像机，其中视频序列跨不同域关联。该计划将解决与AI工程相关的具体研究问题，并开发创新的方法，这些方法在多个领域，最著名的是在DL和计算机视觉中尤其有效。尽管该程序主要集中在基于视频的识别上，但这些模型在各种任务和应用程序中都具有相关性。鉴于DL的最新流行及其众多商业应用，该计划将为与学者合作，与加拿大公司合作，培训HQP，生产高影响力出版物以及启动通过其他授予机制资助的项目提供绝佳的机会。