EAGER: Leveraging Synthetic Data for Visual Reasoning and Representation Learning with Minimal Human Supervision

该项目研究如何使用计算机图形创建的合成数据来开发理解视觉数据的算法。合成数据提供了现实世界图像难以获得的灵活性，并提供了探索仅靠现实世界图像难以解决的问题的机会。该项目开发了用于推理对象遮挡和自监督表示学习的新算法，其中无需人工注释的语义标签的帮助即可开发有用的图像特征。该项目提供了新的算法，有可能使自主系统和安全领域的应用受益。除了科学影响之外，该项目还开展补充性的教育和外展活动，让学生参与研究和 STEM。这项研究探索了从合成数据中学习的新颖算法，以进行视觉推理和表示学习。虽然合成数据的使用在计算机视觉领域有着悠久的历史，但它主要用于补充自然图像数据以解决标准任务。相比之下，该项目使用合成数据在相对未探索的问题上取得进展，在这些问题中，给定的现实世界图像很难获得真实情况。该项目由三个主要目标组成，每个目标都利用了用户可以完全控制合成数据集中发生的所有事情的事实。在 Thrust I 中，它研究了一种使用合成数据进行表示学习的新方法，而在 Thrust II 中，它扩展了算法以区分特定任务和通用功能。最后，在 Thrust III 中，它探索了一种推理对象遮挡的新方法。

This project investigates how synthetic data created using computer graphics can be used for developing algorithms that understand visual data. Synthetic data provides flexibility that is difficult to obtain with real-world imagery, and enables opportunities to explore problems that would be difficult to solve with real-world imagery alone. This project develops new algorithms for reasoning about object occlusions, and for self-supervised representation learning, in which useful image features are developed without the aid of human-annotated semantic labels. The project provides new algorithms that have the potential to benefit applications in autonomous systems and security. In addition to scientific impact, the project performs complementary educational and outreach activities that engage students in research and STEM.This research explores novel algorithms that learn from synthetic data for visual reasoning and representation learning. While the use of synthetic data has a long history in computer vision, it has mainly been used to complement natural image data to solve standard tasks. In contrast, this project uses synthetic data to make advances in relatively unexplored problems, in which ground-truth is difficult to obtain given real-world imagery. The project consists of three major thrusts, each of which exploits the fact that a user has full control of everything that happens in a synthetic dataset. In Thrust I, it investigates a novel approach to representation learning using synthetic data, and in Thrust II, it extends the algorithm to disentangle task-specific and general-purpose features. Finally, in Thrust III, it explores a novel approach for reasoning about object occlusions.