RI:Small:Exploring Efficient Bayesian Model-Augmentation Techniques for Decomposible Contrastive Representation Learning

现代深度学习模型在大量的计算负载下训练网络规模的大数据，在许多现实世界的问题上取得了最先进的性能。从大规模未标记数据中学习良好表示的能力是成功的关键。虽然已经开发了许多方法，但它们的基本性质和局限性仍然没有得到很好的理解。对比表示学习（CRL）是一种学习表示数据的方法，使相似的数据彼此接近，而不相似的数据彼此相隔很远。该项目研究了CRL的局限性，以使其更适合大数据，并适当地应用于计算机视觉等现实世界的问题。该项目还将支持布法罗大学为本科生和研究生提供机器学习相关课程的持续开发，以及让K-12学生接触计算机科学领域的各种外展活动。本研究从贝叶斯原理出发，发展了一个可扩展的条件分解对比学习框架，并将其扩展到联邦学习和多模型学习中。首先，将应用一种增强技术来解耦正负样本的纠缠，从而导致可以用无偏随机梯度优化的条件可分解损失。其次，将进一步改进该技术，为CRL开发一个通信高效的分布式训练框架，其中客户端不再需要与其他客户端显式通信以获取其他负样本。第三，改进后的CRL技术将进一步应用于新兴的视觉语言建模基础模型领域。该项目还将向更广泛的人工智能社区传播共享数据和基准，例如通过Github、演示和研讨会组织。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Modern deep learning models that trains large on web-scale data with a heavy computational load are achieving state-of-the-art performance on many real-world problems. The ability to learning good representations from the large-scale unlabeled data is the key to the success. Although many methods have been developed, their underlying properties and limitations are still not well understood. Contrastive Representation Learning (CRL) is a method of learning to represent the data such that similar data are close to each other while dissimilar data are far apart. This project investigates the limitations of CRL in order to make it more scalable to big data and be appropriately applied to real-world problems such as computer vision. The project will also support the continued development of machine-learning related courses for undergraduate and graduate students at University at Buffalo as well as various outreach activities to expose K-12 students to the field of computer science.This research develops a scalable conditional decomposable contrastive learning framework from the Bayesian principle and extend it to the federated learning and multi-model learning. First, an augmentation technique will be applied to decouple the entanglement of positive-negative samples, leading to a conditional decomposable loss that can be optimized with unbiased stochastic gradients. Second, the technique will be further refined to develop a communication-efficient distributed training framework for CRL, where clients no longer need to explicitly communicate with other clients to fetch other negative samples. Third, the improved CRL technique will be further applied to the emerging field of foundation models for vision-and-language modeling. The project will also result in the dissemination of shared data and benchmarks to the broader AI community, for example through Github, presentation and workshop organization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.