Controllable Image Synthesis

图像合成的任务是自动生成新的人工图像和操纵现有的。至关重要的是，这些图像或其合成修改必须对人类观察者来说是真实的。近年来，机器学习技术继续推进图像合成任务的最新技术-其中最成功的技术是采用深度神经网络实现这些目标的生成对抗网络。然而，大多数现有研究的主要目标是提高这种合成图像的视觉质量，而不是对生成过程有更深入的了解。此外，正如这种“黑匣子”深度神经网络的常见问题一样，解释特定输出的原因或方式并不总是那么简单。因此，能够修改这些网络的方式，以产生一个特定的期望的变化的合成图像是一个重要的任务，已被忽视得多，我们提出的研究将集中在这方面的可控图像合成。我们计划设计新的方法，分解和算法，以更好地理解和解释这些网络的内部机制和表示-引导我们开发方法，以更好地控制图像合成过程的创意目的，并为其他下游任务。我们提出的研究应用程序与EPSRC的“图像和视觉计算”的研究领域高度一致。特别是，我们希望我们开发的技术“与创意产业高度相关”-例如，这些技术可以用于自动化电影后期制作，或摄影师执行的常见编辑任务。

The task of image synthesis is concerned with automatically generating new artificial images and manipulating existing ones. Crucially, these images or synthetic modifications thereof must appear to be realistic to human observers. Over the recent years, machine learning techniques have continued to advance the state-of-the-art for this task of image synthesis--the most successful technique of which is the Generative Adversarial Networks that employ deep neural networks to achieve these ends. The majority of existing research however has had a primary goal of improving the visual quality of such synthetic images, and not of developing a deeper understanding of the generation process. Moreover, as is a common problem with such "black box" deep neural networks, interpreting why or how a particular output was produced is not always so straightforward. Thus, the ability to modify these networks in such a way as to produce a particular desired change to the synthetic images is an important task that has been much more neglected.Our proposed research will focus on this aspect of controllable image synthesis. We plan to design new methodologies, decompositions, and algorithms to better understand and interpret the inner mechanisms and representations of these networks--leading us to develop ways to better control the image synthesis process for creative ends, and for other downstream tasks.Applications of our proposed research are highly aligned with EPSRC's research area of "Image and vision computing". In particular, we expect the techniques we develop to have "a high degree of relevance to the creative industries"--for example, such techniques could be utilised to automate film post-production, or common editing tasks performed by photographers.