RI: Small: Novel Generative Models for High-Diversity Visual Speculation

RI：小型：用于高多样性视觉推测的新颖生成模型

• 批准号: 1718221
• 负责人: Alexander Schwing
• 金额: $ 45万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1718221&HistoricalAwards=false
• 关键词: RI; Small; Novel; Generative; Models

The amount of available data and its dimensionality is soaring, and by now, images are one of the most widely used modalities for sharing impressions. This makes easy-to-use image editing capabilities increasingly important. However present day editing techniques are either very simple or designed for expert users which have a detailed understanding of image properties. This project contributes to an understanding of easy to use algorithms for complex image editing, which we refer to as visual speculation. Examples include attribute transformation (change a scene recorded in summer to what it could have looked like during winter), colorization (producing a color image from a monochrome input), and reshading (changing the illumination of the image). These tasks have numerous applications by producing controllable and photorealistic images for different purposes such as virtual/augmented reality content generation. The project integrated with education through curriculum development and supporting graduate/undergraduate student research.This research studies methods for visual speculation, i.e., applying complex image editing tasks which modify existing pictures to fit the users? desire while looking real. Examples include: attribute transformation, colorization, and reshading. In each case, solutions are widely ambiguous and good methods generate a diverse range of plausible solutions while offering control to the user. These requirements pose challenges for existing methods which struggle to provide diversity while retaining control. For example, straightforward conditioning based on control variables causes artifacts because the amount of data per instance is no longer sufficient. To alleviate those issues, the proposed work develops high-diversity, high-quality generation models by augmenting state-of-the-art deep generative machinery with image representations that expose data sharing opportunities. The insights obtained from this work result in image editing capabilities that provide high-level control mechanisms.

可用数据的数量及其维度正在飙升，到目前为止，图像是最广泛使用的共享印象的方式之一。这使得易于使用的图像编辑功能变得越来越重要。然而，目前的编辑技术要么非常简单，要么是为对图像属性有详细了解的专家用户设计的。这个项目有助于理解复杂图像编辑的易于使用的算法，我们称之为视觉推测。示例包括属性转换（将夏季记录的场景更改为冬季可能的样子）、着色（从单色输入生成彩色图像）和重塑（更改图像的照明）。这些任务通过为不同目的（如虚拟/增强现实内容生成）生成可控和逼真的图像有许多应用。该项目通过课程开发和支持研究生/本科生的研究与教育相结合。本研究研究了视觉推测的方法，即应用复杂的图像编辑任务，修改现有图片以适应用户？欲望，同时看起来真实。示例包括：属性转换、着色和重塑。在每种情况下，解决方案都是非常模糊的，而好的方法在向用户提供控制的同时，会产生各种各样的合理解决方案。这些要求对现有的方法提出了挑战，这些方法难以在保持控制的同时提供多样性。例如，基于控制变量的直接条件反射会导致工件，因为每个实例的数据量不再足够。为了缓解这些问题，建议的工作通过增加最先进的深度生成机器和暴露数据共享机会的图像表示来开发高多样性、高质量的生成模型。从这项工作中获得的见解产生了提供高级控制机制的图像编辑功能。