Generative models for 3D textures

3D纹理是2D光栅纹理的体积等效物。与2D对应物相比，它们具有几个优势。然而，尽管最近取得了进展，并且与2D纹理不同，允许建模和创建此类纹理的专用技术是有限的。与扫描2D图像不同，体积颜色信息的数字化通常是不切实际的。这就产生了对高效可控合成算法的需求。一个原因是缺乏直观的规范模型，允许用户指出他的想法。另一个原因是缺乏可扩展的模型，可以处理高分辨率的三维。我们的主要研究目标是追求基础和应用研究，为开发新的表示和设计算法的三维纹理。我们有四个具体目标。1)以用户为中心的设计模型，用于控制3D纹理合成。首先，我们将提供高度可控的合成方法。这提出了一个主要问题，特别是在处理具有可区分图案的纹理时。与2D合成不同，用户可以提供一个小例子来输入算法，提供3D例子是不切实际的。我们将使用条件对抗网络提出新的控制措施。2)丰富3D打印的结构声音模型。3D打印提供了一个具有挑战性的环境，使设计师和艺术家有一个令人兴奋的机会来想象和生产复杂的图案和物体。我们将提出一种方法，解决基于打印的3D纹理的对象的结构健全性。3)视频风格转换的时间相干模型。这是自动将一个视频的内容和图像的风格组合成风格化的输出视频的任务。通常，艺术图像的风格被转移到照片般逼真的视频。我们建议使用一个3D卷积神经网络来处理本地视频块，一次几帧。在纹理尺度上，这将保证空间和时间维度的一致性，避免估计光流。4)边缘感知模型在医疗应用中的数据扩充.深度学习技术在医学图像分析中取得了成功。然而，需要大的标记训练数据库来实现良好的性能。这些数据通常很难收集，因为它通常需要手动标记。我们将评估如何通过3D约束纹理合成数据增强可以提高性能。我们将专注于依赖于纹理特征的模式，如组织学和MRI。拟议的研究计划将开发工具和技术，使突破性创新，导致在几个应用程序的重大进展，有关加拿大计算机动画和视频游戏行业，以及医疗领域。

3D textures are the volumetric equivalent of 2D raster textures. They offer several advantages over their 2D counterparts. However, despite recent progress and unlike 2D textures, dedicated techniques that allow to model and create such textures are limited. Unlike scanning a 2D image, digitization of volumetric color information is usually impractical. This creates the need for efficient and controllable synthesis algorithms. One reason is the lack of intuitive specification models allowing a user to indicate what he has in mind. Another reason is the lack of scalable models that can handle three dimensions at high resolution. Our main research objective is to pursue fundamental and applied research for the development of novel representations and design algorithms of 3D textures. We target four specific objectives. 1) User centered design models for controlling 3D textures synthesis. First, we will offer a highly controllable synthesis method. This raises a major issue, especially when tackling textures with distinguishable patterns. Unlike 2D synthesis, where the user can provide a small example to input the algorithm, providing 3D examples is impractical. We will propose novel controls using a conditionnal adversarial network. 2) Structural sound models for enriching 3D printing. 3D printing offers a challenging context allowing designers and artists with an exciting opportunity to imagine and produce complex patterns and objects. We will propose a method that tackles structural soundness for printed 3D textures based objects. 3) Temporal coherent models for video style transferring. This is the task of automatically combining the content of one video and the style of an image into a stylized output video. Typically, the style of an artistic image is transferred to a photo-realistic video. We propose to use a 3D convolutional neural network to stylize local video blocks, several frames at a time. At the texture scale, this will guarantee consistency in both spatial and temporal dimensions, avoiding the estimation of the optical flow. 4) Edge--aware models for data- augmentation in medical applications. Deep learning techniques have been successful in medical image analysis. However, large labeled training databases are needed to achieve good performance. Such data is usually difficult to gather since it usually needs manual labeling. We will evaluate how data augmentation through 3D constrained texture synthesis can improve performances. We will focus on modalities relying on texture features such as histology and MRI. The proposed research program will develop tools and techniques enabling breakthrough innovation, leading to major advances in several applications, related to the Canadian computer animation and video games industry, and the medical field.