Generative models for 3D textures

3D纹理是2D栅格纹理的体积等效体。与2D同类产品相比，它们提供了几个优势。然而，尽管最近取得了进展，但与2D纹理不同的是，允许建模和创建此类纹理的专用技术是有限的。与扫描2D图像不同，体积颜色信息的数字化通常是不切实际的。这就产生了对高效且可控的合成算法的需求。一个原因是缺乏直观的规范模型，允许用户指明他心中的想法。另一个原因是缺乏能够以高分辨率处理三维的可扩展模型。我们的主要研究目标是为开发新的三维纹理表示和设计算法提供基础和应用研究。我们的目标是四个具体目标。1)以用户为中心的控制3D纹理合成的设计模型。首先，我们将提供一种高度可控的合成方法。这带来了一个主要问题，特别是在处理具有可区分图案的纹理时。与2D合成不同的是，用户可以提供一个小示例来输入算法，而提供3D示例是不切实际的。我们将提出使用条件对抗网络的新控制措施。2)丰富3D打印的结构健全模型。3D打印提供了一个具有挑战性的环境，使设计师和艺术家有一个令人兴奋的机会来想象和制作复杂的图案和物体。我们将提出一种方法来解决基于打印3D纹理的对象的结构可靠性问题。3)视频风格迁移的时间相关模型。这是自动将一个视频的内容和图像的样式组合成风格化的输出视频的任务。通常，艺术图像的风格被转换为照片级的视频。我们建议使用3D卷积神经网络来样式化局部视频块，一次几帧。在纹理尺度上，这将保证在空间和时间维度上的一致性，避免了对光流的估计。4)用于医疗应用中数据增强的边缘感知模型。深度学习技术在医学图像分析中已经取得了成功。然而，为了获得良好的性能，需要大量的带标签的训练数据库。这种数据通常很难收集，因为它通常需要手动标记。我们将评估通过3D约束纹理合成进行数据增强如何提高性能。我们将重点介绍依赖于组织学和核磁共振等纹理特征的模式。拟议的研究计划将开发能够实现突破性创新的工具和技术，导致与加拿大计算机动画和视频游戏行业以及医疗领域相关的几个应用程序取得重大进展。

3D textures are the volumetric equivalent of 2D raster textures. They offer several advantages over their 2D counterparts. However, despite recent progress and unlike 2D textures, dedicated techniques that allow to model and create such textures are limited. Unlike scanning a 2D image, digitization of volumetric color information is usually impractical. This creates the need for efficient and controllable synthesis algorithms. One reason is the lack of intuitive specification models allowing a user to indicate what he has in mind. Another reason is the lack of scalable models that can handle three dimensions at high resolution. Our main research objective is to pursue fundamental and applied research for the development of novel representations and design algorithms of 3D textures. We target four specific objectives. 1) User centered design models for controlling 3D textures synthesis. First, we will offer a highly controllable synthesis method. This raises a major issue, especially when tackling textures with distinguishable patterns. Unlike 2D synthesis, where the user can provide a small example to input the algorithm, providing 3D examples is impractical. We will propose novel controls using a conditionnal adversarial network. 2) Structural sound models for enriching 3D printing. 3D printing offers a challenging context allowing designers and artists with an exciting opportunity to imagine and produce complex patterns and objects. We will propose a method that tackles structural soundness for printed 3D textures based objects. 3) Temporal coherent models for video style transferring. This is the task of automatically combining the content of one video and the style of an image into a stylized output video. Typically, the style of an artistic image is transferred to a photo-realistic video. We propose to use a 3D convolutional neural network to stylize local video blocks, several frames at a time. At the texture scale, this will guarantee consistency in both spatial and temporal dimensions, avoiding the estimation of the optical flow. 4) Edge--aware models for data- augmentation in medical applications. Deep learning techniques have been successful in medical image analysis. However, large labeled training databases are needed to achieve good performance. Such data is usually difficult to gather since it usually needs manual labeling. We will evaluate how data augmentation through 3D constrained texture synthesis can improve performances. We will focus on modalities relying on texture features such as histology and MRI. The proposed research program will develop tools and techniques enabling breakthrough innovation, leading to major advances in several applications, related to the Canadian computer animation and video games industry, and the medical field.