Deep Learning with Little Labelled Data

机器学习(ML)是人工智能(AI)的一个领域，旨在通过根据观察和经验推断模型来为计算机提供学习能力。人工智能最近的许多进展都源于ML，特别是深度学习(DL)的子领域。DL的目标是学习分层结构，以便从非结构化的原始数据(例如，图像、声音、文本)中提取特征，从而使许多经过充分研究的任务(例如，对象识别、自动翻译)的水平得到显著改进。然而，这些技术需要大量的数据，需要庞大的带注释的数据集才能有效地学习如何完成某些特定的任务。虽然这样的数据集可用于某些特定的已建立的任务，但为新的应用程序收集和注释足够大的数据集可能是昂贵的、成本和时间方面的。我们提出了一个研究计划，研究各种方法来提高ML和DL的效率，以解决只有较小的标注数据集可用的问题。我们希望建立在其他相关或卫星数据集的基础上，这些数据集具有相同类型的数据，但其注释(如果存在)不适用于这项任务。从中，我们将通过学习(或微调)带有与任务相关的注释的小数据集，为当前任务生成一个工作模型。该计划围绕四个目标进行组织：1)研究如何通过利用从不同任务提取的知识来更好地学习模型，这些知识可以重复使用或适应当前上下文；2)改进元学习方法，使其能够仅从少量样本中学习新概念；3)开发新的方法，通过非监督学习提取更好的通用表示；4)将新方法应用于实际应用，以评估所提出方法的实用性，并在应用领域本身取得有意义的贡献。在本研究计划中研究的方法有可能使ML和DL在各种情况下可用，在这些情况下，没有可用的大数据集很好地适应手头的特定任务。这些基于ML的系统将通过利用类似的历史任务来帮助人类快速适应给定的任务；它们将具有为每个用户快速定制的能力。我们的目标是学习一些可用于各种目的的模式(例如，图像、文本、语音)的更好表示。我们将寻找发现机制，让人工智能适应新的学习任务，能够通过几个例子提高性能。最后，我们将把我们的技术应用到黑盒优化、超分辨率显微镜和视觉位置识别等领域。

Machine Learning (ML) is a field of Artificial Intelligence (AI) that aims at providing computers with learning abilities, by inferring models from observations and experience. Many of the recent advances of AI have stemmed from ML, in particular the subfield of Deep Learning (DL). DL aims at learning hierarchical structures, in order to extract features from unstructured raw data (e.g., images, sound, text), allowing significant improvements over the state-of-the-art for many well-studied tasks (e.g., object recognition, automatic translation). However, these techniques are data hungry, requiring huge annotated datasets to be able to efficiently learn how to accomplish some specific tasks. Although such datasets are available for some specific well-established tasks, collecting and annotating sufficiently large datasets for a novel application can be expensive, cost- and time-wise. We propose a research program investigating various approaches to improve the efficiency of ML and DL for solving problems where only small annotated datasets available. We are looking to build upon other related or satellite datasets having the same kind of data, but whose annotations, if they exist, are not adapted to the task. From it, we would produce a working model for our current task, by learning (or fine-tuning) on the small datasets with the task-related annotations. The program is organized around four objectives: 1) To investigate how to better learn models by exploiting knowledge extracted from different tasks, which can be reused or adapted to the current context; 2) To improve meta-learning methods that can learn new concepts from only few samples; 3) To develop new approaches to extract better general representations through unsupervised learning; 4) To apply the novel methods to real-world applications, in order to both assess the practicality of the proposed methods and achieve meaningful contributions in the application domain themselves. Methods investigated within this research program have the potential of making ML and DL usable in a variety of contexts where there are no available big datasets well adapted to the specific task at hand. These ML-based systems will assist humans to adapt quickly to a given task by exploiting similar historical tasks; they will have the capacity for a rapid customization to each user. We are aiming at learning better representations of some modalities (e.g., image, text, speech), usable for a variety of purposes. We will look for discovering mechanisms for AI to adapt to new learning tasks, being able to ramp up performance with only a few examples. Finally, we will apply our techniques to domains such as black-box optimization, super-resolution microscopy, and visual place recognition.