RI: Small: Novel Generative Models for High-Diversity Visual Speculation

可用的数据量及其维度正在飙升，到目前为止，图像是最广泛使用的印象分享方式之一。这使得易于使用的图像编辑功能变得越来越重要。然而，目前的编辑技术要么非常简单，要么专为对图像属性有详细了解的专家用户设计。这个项目有助于理解复杂图像编辑的易于使用的算法，我们将其称为视觉推测。例如，属性变换(将夏季拍摄的场景更改为冬季时的样子)、彩色化(从单色输入生成彩色图像)和重着色(更改图像的照明)。这些任务通过为诸如虚拟/增强现实内容生成等不同目的产生可控和照片逼真的图像而具有许多应用。该项目通过课程开发与教育相结合，并支持研究生/本科生的研究。本研究研究视觉推测的方法，即应用复杂的图像编辑任务来修改现有图片以适应用户？看起来真实的同时也有欲望。示例包括：属性变换、彩色化和重着色。在每一种情况下，解决方案都是非常模糊的，好的方法在为用户提供控制的同时，会生成各种合理的解决方案。这些要求对现有方法提出了挑战，这些方法难以在保持控制的同时提供多样性。例如，基于控制变量的直接条件会产生伪像，因为每个实例的数据量不再足够。为了缓解这些问题，拟议的工作开发了高多样性、高质量的生成模型，方法是使用暴露数据共享机会的图像表示来增强最先进的深度生成机制。从这项工作中获得的见解导致了提供高级控制机制的图像编辑能力。

The amount of available data and its dimensionality is soaring, and by now, images are one of the most widely used modalities for sharing impressions. This makes easy-to-use image editing capabilities increasingly important. However present day editing techniques are either very simple or designed for expert users which have a detailed understanding of image properties. This project contributes to an understanding of easy to use algorithms for complex image editing, which we refer to as visual speculation. Examples include attribute transformation (change a scene recorded in summer to what it could have looked like during winter), colorization (producing a color image from a monochrome input), and reshading (changing the illumination of the image). These tasks have numerous applications by producing controllable and photorealistic images for different purposes such as virtual/augmented reality content generation. The project integrated with education through curriculum development and supporting graduate/undergraduate student research.This research studies methods for visual speculation, i.e., applying complex image editing tasks which modify existing pictures to fit the users? desire while looking real. Examples include: attribute transformation, colorization, and reshading. In each case, solutions are widely ambiguous and good methods generate a diverse range of plausible solutions while offering control to the user. These requirements pose challenges for existing methods which struggle to provide diversity while retaining control. For example, straightforward conditioning based on control variables causes artifacts because the amount of data per instance is no longer sufficient. To alleviate those issues, the proposed work develops high-diversity, high-quality generation models by augmenting state-of-the-art deep generative machinery with image representations that expose data sharing opportunities. The insights obtained from this work result in image editing capabilities that provide high-level control mechanisms.