High-quality Shape Synthesis with User-guided Deep Neural Networks

我的研究计划的长期目标是开发促进计算机图形内容创建的方法，特别是3D形状。3D内容在各种应用程序中都很重要，例如电脑游戏、电影和动画以及建筑模拟。在下一期资金中，我建议开发使用深度神经网络(DNN)合成3D形状的计算方法，其中合成可以由用户控制，生成的形状质量高。然后可以应用这些方法来促进不同类型的3D内容的生成。内容创建具有挑战性，因为高质量的3D形状通常是通过对形状的几何图形进行显式建模来创建的。这一过程涉及熟练艺术家在需要大量培训的耗时过程中使用复杂的用户界面。因此，在过去的二十年中，计算机图形学研究也提出了便于非专家用户对3D形状进行建模的方法，例如参数模型和绘制界面。然而，这些方法中的许多方法要么仍然需要用户足够的艺术技能，要么需要大量的手工工作来进行数据的预处理。近年来，基于机器学习的3D内容合成方法引起了人们的极大兴趣，特别是基于深度神经网络的方法，因为深度神经网络与传统方法相比具有一些优点，例如更容易的训练数据准备，不需要手工提取特征方法，高度的泛化能力，以及允许用户合成类似于训练数据的新数据的生成模型。但是，DNN目前具有某些限制，使其无法轻松用于形状建模。在此背景下，我下一期资助的研究计划的目标是研究用DNN改进3D形状的分析和合成的解决方案。更详细地说：(1)我们建议开发方法，使用户能够在使用DNN生成3D形状时进行更直接的控制，以便用户能够根据他们的目标设计形状；(2)我们建议使用与最新技术相比能够生成更高质量的形状的表示，例如基于过程模型的表示，该表示以较低的复杂性生成可编辑的形状；(3)我们建议使用过程模型来生成合成数据，这些数据可以用于使用较少的人工准备数据来训练形状分析DNN，或者用于在受控环境中评估基于DNN的方法。拟议工作的意义在于，所开发的解决方案将能够以较少的人工工作来指导合成高质量的形状。这将产生新的软件工具，改善需要形状建模的行业中目前的做法，使这项技术的用户能够以更低的成本创造出前所未有的多样性和数量的内容。

The long-term goal of my research program is to develop methods for facilitating the creation of computer graphics content, especially 3D shapes. 3D content is important in a variety of applications, such as computer games, movies and animation, and architectural simulations. In the next period of funding, I propose to develop computational methods for synthesizing 3D shapes with deep neural networks (DNNs), where the synthesis can be controlled by users and the generated shapes are of high-quality. These methods can then be applied to facilitate the generation of different types of 3D content. Content creation is challenging since high-quality 3D shapes are commonly created by explicitly modeling the geometry of the shapes. This process involves the use of complex user interfaces by skilled artists in a time-consuming process requiring substantial training. Thus, during the last two decades, computer graphics research has also proposed approaches for facilitating the modeling of 3D shapes by non-expert users, such as parametric models and sketching interfaces. However, many of these methods either still require sufficient artistic skills from the users, or require considerable manual work for pre-processing of the data. In recent years, methods for synthesizing 3D content based on machine learning have sparked much interest, especially methods based on deep neural networks (DNNs), since DNNs offer several advantages over traditional methods, such as easier training data preparation, no need to handcraft feature extraction methods, high generalization capabilities, and generative models that allow users to synthesize new data resembling the training data. However, DNNs currently have certain limitations that prevent them from being easily used for shape modeling. In this context, the goal of my research program for the next period of funding is to investigate solutions for improving the analysis and synthesis of 3D shapes with DNNs. In more detail: (1) We propose to develop methods for enabling more direct user control in the generation of 3D shapes with DNNs, so that users are able to design shapes according to their goals; (2) We propose to use representations that allow to generate higher-quality shapes compared to the state-of-the-art, such as representations based on procedural models that generate editable shapes with low complexity; (3) We propose to use procedural models to generate synthetic data, which can be used for training shape analysis DNNs with less manually-prepared data, or for evaluating DNN-based methods in controlled settings. The significance of the proposed work is that the developed solutions will enable the guided synthesis of high-quality shapes with less manual work. This will result in new software tools that will improve the current practices in industries that require shape modeling, allowing users of this technology to create a diversity and volume of content never seen before with reduced costs.