Broader Self-Supervised Learning with applications in anomaly detection, tabular data, and visual reinforcement learning

点击翻译按钮获取中文摘要

Deep Learning methods have enabled massive progress to be made on challenges that seemed far out of reach just a few years ago, for example, effective and commercially viable object recognition, speech recognition, and machine translation. Many of these breakthroughs have been based on the supervised learning paradigm. Despite their great success these systems are imperfect and do not work in all cases, furthermore, a variety of applications have yet to be successfully addressed (e.g. tabular data). Fortunately, it has been repeatedly demonstrated that larger datasets and models can further improve performance. To scale deep learning methods to a wider variety of applications with economic and social benefits, the need for labels is a limiting factor. In recent years the unsupervised learning paradigm, long believed to be critical to the success of AI, has begun to yield promising results. Specifically, self-supervised learning, where unlabeled input data is used to form targets for supervised learning paradigms has shown to yield reusable representations for a variety of downstream tasks. However, a number of challenges remain in making self-supervised representation learning more broadly useful. This approach relies on a number of highly domain-specific algorithm constructions (e.g. image specific data-augmentation) and has been shown to yield features useful for specific downstream tasks (e.g. image classification) but not useful for others (e.g. reinforcement learning in visual environments). In the proposed 3-year research project we will develop new approaches that will aim to make these methods applicable to a new set of domains and tasks. We will target a key set of application areas including anomaly detection, tabular data, and visual reinforcement learning.