EAGER: Visual Representation Learning Using Mixed Labeled and Unlabeled Data

EAGER：使用混合标记和未标记数据的视觉表示学习

• 批准号: 1845216
• 负责人: Hamed Pirsiavash
• 金额: $ 16.7万
• 依托单位: University of Maryland Baltimore County
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1845216&HistoricalAwards=false
• 关键词: EAGER; Visual; Representation; Learning; Using

Recent advances in deep learning has led to great results in visual recognition and object detection. These deep learning models have various applications from self-driving cars to early disease diagnosis and household robots. However, most such models are supervised, meaning that they need large scale manually annotated datasets to tune the parameters, and obtaining the annotation may be expensive in many applications. This project explores a family of self-supervised learning algorithms where the learning is based on unlabeled data only. The new models can learn visual features that can be used for various visual recognition tasks including object detection and action recognition. This project provides research opportunities for under-represented groups and integrates research outcomes into the course curriculum.This project studies a family of self-supervised learning algorithms that can learn rich features from unlabeled images and videos. Self-supervised learning algorithms harvest the knowledge from unlabeled data by modeling some regularity in the space of natural images or videos. This project studies novel self-supervised learning algorithms based on constraining the learning by relating transformations of images to transformations of their representations. Moreover, this project studies a novel multi-task learning framework for aggregating the knowledge learned from multiple supervised and self-supervised learning algorithms. This algorithm uses quantization methods to ignore the task specific details of the representation in transferring the knowledge. This algorithm results in a rich set of representations that generalize well across various visual recognition tasks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度学习的最新进展在视觉识别和对象检测方面取得了巨大成果。这些深度学习模型具有各种应用，从自动驾驶汽车到早期疾病诊断和家用机器人。然而，大多数这样的模型是有监督的，这意味着它们需要大规模手动注释的数据集来调整参数，并且在许多应用中获得注释可能是昂贵的。该项目探索了一系列自监督学习算法，其中学习仅基于未标记的数据。新模型可以学习可用于各种视觉识别任务的视觉特征，包括对象检测和动作识别。该项目为代表性不足的群体提供研究机会，并将研究成果整合到课程中。该项目研究了一系列自监督学习算法，可以从未标记的图像和视频中学习丰富的特征。自监督学习算法通过对自然图像或视频空间中的一些规律进行建模，从未标记数据中获取知识。该项目研究了新的自监督学习算法，该算法基于通过将图像的变换与其表示的变换相关联来约束学习。此外，该项目研究了一种新的多任务学习框架，用于聚合从多个监督和自监督学习算法中学习到的知识。该算法使用量化方法，忽略了任务的具体细节的表示在转移的知识。这个奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

MASTAF: A Model-Agnostic Spatio-Temporal Attention Fusion Network for Few-shot Video Classification

• DOI: 10.1109/wacv56688.2023.00254
• 发表时间: 2021-12
• 期刊: 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
• 作者: Huan Zhang;H. Pirsiavash;Xin Liu

Defending Against Patch-based Backdoor Attacks on Self-Supervised Learning

• DOI: 10.1109/cvpr52729.2023.01178
• 发表时间: 2023-04
• 期刊: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Ajinkya Tejankar;Maziar Sanjabi;Qifan Wang;Sinong Wang;Hamed Firooz;H. Pirsiavash;L Tan

Hidden Trigger Backdoor Attacks

• DOI: 10.1609/aaai.v34i07.6871
• 发表时间: 2019-09
• 期刊: ArXiv
• 作者: Aniruddha Saha;Akshayvarun Subramanya;H. Pirsiavash

Backdoor Attacks on Self-Supervised Learning

• DOI: 10.1109/cvpr52688.2022.01298
• 发表时间: 2021-05
• 期刊: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Aniruddha Saha;Ajinkya Tejankar;Soroush Abbasi Koohpayegani;H. Pirsiavash

Adaptive Inverse Transform Sampling For Efficient Vision Transformers

• 发表时间: 2022
• 作者: Mohsen Fayyaz-;Soroush Abbasi Koohpayegani;F. Jafari;Sunando Sengupta;Hamid Reza Vaezi Joze;Eric Sommerlade