III: Small: Transfer Learning using Transformation among Models and Samples

随着大量未标记数据的生成并在许多领域中可用，在这些领域中标注数据变得繁重，并在维护机器学习数据库方面造成了主要瓶颈。这个项目将调查一系列迁移学习方法，作为一种自动注释工具，在没有人工参与的情况下，为各种机器学习环境注释数据。该项目的迁移学习方法的新颖性是基于基于匹配的优化技术的共同概念来解决不同形式的迁移学习。将使用针对不同形式的不同级别的变换来进行优化。计划中的迁移学习框架将利用目标领域中大量未标记的数据或少数标记的数据，以及源域中标记的源数据、源模型或其他辅助信息形式的先验知识。使用这种常见的基于匹配的优化框架，这将为不同形式的迁移学习带来从低级、基于样本的匹配到高级、基于模型的匹配的自然过渡。这一系列的迁移学习方法将在智能机器人和自动驾驶汽车等不同领域产生良好的影响，使它们在新的和不断变化的环境中高效运行，而不需要在新的环境中使用大量的注释数据。本项目将研究两种主要的迁移学习形式--域自适应和少机会学习。本研究将重点研究提出的基于匹配的优化技术解决不同形式的迁移学习的效果。该项目将侧重于三项主要任务，视每项任务提供的信息而定：(任务1)无监督的领域调整，即对源域数据加标签，而对目标域数据不加标签。在这种情况下，项目团队将通过将每个源域样本与每个目标域样本进行匹配来研究优化，以学习可概化的目标模型；(任务2)假设迁移学习，其中源和目标域任务不同，仅使用源模型和稀疏标记的目标域数据来学习可概化的目标模型。模型将通过源模型和目标领域样本之间的匹配来学习；(任务3)少机会学习，其目标是通过利用辅助源知识从目标领域中的几个标记样本中学习一个可概括的目标模型。项目团队将研究源-模型参数之间的转换是否可以替代为有用的辅助源域知识。因此，计划中的研究将最小化机器学习中使用的大量标记样本的需求，并将实现可跨任务和领域推广的健壮学习系统。此外，由于匹配是在本地和明确地在每个单独的样本/信息模型之间进行的，因此结果预计会比以前的方法更好。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

As huge volumes of unlabeled data are generated and made available in many domains, annotating data in these domains becomes burdensome and creates a major bottleneck in maintaining machine-learning databases. This project will investigate a family of transfer-learning methods as an automatic annotation tool, without human involvement, in annotating data for various machine-learning settings. The novelty of the project's transfer-learning approach is based on the common concept of matching-based optimization technique to solve the different forms of transfer learning. The optimization will be carried out using transformations at different levels for different forms. The planned transfer-learning framework will exploit lots of unlabeled data or a few labelled data in the target domain and prior knowledge in the form of labelled source data, source models or other auxiliary information in the source domain. Using this common matching-based optimization framework, this will bring out a natural transition from low-level, sample-based matching to high-level, model-based matching for the different forms of transfer learning. The family of transfer learning methods will have promising ramifications in diverse areas such as intelligent robots and self-driving cars so that they operate efficiently in new and changing environments without the need of large amount of annotated data in the new environments.This project will investigate two major forms of transfer learning -- domain adaptation and few-shot learning. The research will focus on studying the effect of the proposed matching-based optimization technique to solve the different forms of transfer learning. The project will focus on three major tasks, depending on what information is available in each task: (Task 1) Unsupervised domain adaptation, where the source-domain data is labelled while the target-domain data is unlabeled. In this case, the project team will investigate the optimization based on matching each source-domain sample with each target-domain sample to learn a generalizable target model; (Task 2) Hypothesis transfer learning, where the source and the target domain tasks are different, and only source models and sparsely labelled target domain data will be used to learn a generalizable target model. The model will be learned using matching between source models and target-domain samples; (Task 3) Few-shot learning, where the goal is to learn a generalizable target model from a few labelled samples in the target domain by utilizing auxiliary source knowledge. The project team will study whether transformation between source-model parameters can be substituted as useful auxiliary source-domain knowledge. Hence, the planned research will minimize the requirement of obtaining lots of labelled samples used in machine learning, and it will realize robust learning systems that are generalizable across tasks and domains. Furthermore, since the matching is carried out among each individual sample/model of information locally and explicitly, the results are expected to be better than previous methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.