Broader Self-Supervised Learning with applications in anomaly detection, tabular data, and visual reinforcement learning

深度学习方法已经在几年前似乎遥不可及的挑战上取得了巨大进展，例如，有效且商业上可行的物体识别、语音识别和机器翻译。许多这些突破都是基于监督学习范式的。尽管取得了巨大的成功，但这些系统并不完美，并不是在所有情况下都能工作，此外，各种应用尚未成功解决（例如表格数据）。幸运的是，事实一再证明，更大的数据集和模型可以进一步提高性能。为了将深度学习方法扩展到具有经济和社会效益的更广泛的应用中，对标签的需求是一个限制因素。近年来，长期以来被认为对人工智能成功至关重要的无监督学习范式已经开始产生有希望的结果。特别是自监督学习，其中未标记的输入数据被用来形成监督学习范式的目标，已经显示为各种下游任务产生可重用的表示。然而，要使自监督表示学习更广泛地发挥作用，仍然存在许多挑战。这种方法依赖于许多高度领域特定的算法结构（例如图像特定的数据增强），并已被证明可以产生对特定下游任务有用的特征（例如图像分类），但对其他任务无用（例如视觉环境中的强化学习）。在提议的3年研究项目中，我们将开发新的方法，旨在使这些方法适用于一组新的领域和任务。我们将针对一组关键的应用领域，包括异常检测、表格数据和视觉强化学习。

Deep Learning methods have enabled massive progress to be made on challenges that seemed far out of reach just a few years ago, for example, effective and commercially viable object recognition, speech recognition, and machine translation. Many of these breakthroughs have been based on the supervised learning paradigm. Despite their great success these systems are imperfect and do not work in all cases, furthermore, a variety of applications have yet to be successfully addressed (e.g. tabular data). Fortunately, it has been repeatedly demonstrated that larger datasets and models can further improve performance. To scale deep learning methods to a wider variety of applications with economic and social benefits, the need for labels is a limiting factor. In recent years the unsupervised learning paradigm, long believed to be critical to the success of AI, has begun to yield promising results. Specifically, self-supervised learning, where unlabeled input data is used to form targets for supervised learning paradigms has shown to yield reusable representations for a variety of downstream tasks. However, a number of challenges remain in making self-supervised representation learning more broadly useful. This approach relies on a number of highly domain-specific algorithm constructions (e.g. image specific data-augmentation) and has been shown to yield features useful for specific downstream tasks (e.g. image classification) but not useful for others (e.g. reinforcement learning in visual environments). In the proposed 3-year research project we will develop new approaches that will aim to make these methods applicable to a new set of domains and tasks. We will target a key set of application areas including anomaly detection, tabular data, and visual reinforcement learning.