High-quality Shape Synthesis with User-guided Deep Neural Networks

通过用户引导的深度神经网络进行高质量形状合成

• 批准号: RGPIN-2022-04903
• 负责人: vanKaick, Oliver
• 金额: $ 2.11万
• 依托单位: Carleton University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=750391
• 关键词: quality; Shape; Synthesis; User; guided

The long-term goal of my research program is to develop methods for facilitating the creation of computer graphics content, especially 3D shapes. 3D content is important in a variety of applications, such as computer games, movies and animation, and architectural simulations. In the next period of funding, I propose to develop computational methods for synthesizing 3D shapes with deep neural networks (DNNs), where the synthesis can be controlled by users and the generated shapes are of high-quality. These methods can then be applied to facilitate the generation of different types of 3D content. Content creation is challenging since high-quality 3D shapes are commonly created by explicitly modeling the geometry of the shapes. This process involves the use of complex user interfaces by skilled artists in a time-consuming process requiring substantial training. Thus, during the last two decades, computer graphics research has also proposed approaches for facilitating the modeling of 3D shapes by non-expert users, such as parametric models and sketching interfaces. However, many of these methods either still require sufficient artistic skills from the users, or require considerable manual work for pre-processing of the data. In recent years, methods for synthesizing 3D content based on machine learning have sparked much interest, especially methods based on deep neural networks (DNNs), since DNNs offer several advantages over traditional methods, such as easier training data preparation, no need to handcraft feature extraction methods, high generalization capabilities, and generative models that allow users to synthesize new data resembling the training data. However, DNNs currently have certain limitations that prevent them from being easily used for shape modeling. In this context, the goal of my research program for the next period of funding is to investigate solutions for improving the analysis and synthesis of 3D shapes with DNNs. In more detail: (1) We propose to develop methods for enabling more direct user control in the generation of 3D shapes with DNNs, so that users are able to design shapes according to their goals; (2) We propose to use representations that allow to generate higher-quality shapes compared to the state-of-the-art, such as representations based on procedural models that generate editable shapes with low complexity; (3) We propose to use procedural models to generate synthetic data, which can be used for training shape analysis DNNs with less manually-prepared data, or for evaluating DNN-based methods in controlled settings. The significance of the proposed work is that the developed solutions will enable the guided synthesis of high-quality shapes with less manual work. This will result in new software tools that will improve the current practices in industries that require shape modeling, allowing users of this technology to create a diversity and volume of content never seen before with reduced costs.

我的研究计划的长期目标是开发促进计算机图形内容，特别是3D形状的创建的方法。3D内容在各种应用程序中非常重要，例如计算机游戏、电影和动画以及建筑模拟。在下一个资助期间，我建议开发使用深度神经网络（DNN）合成3D形状的计算方法，其中合成可以由用户控制，并且生成的形状具有高质量。然后可以应用这些方法来促进不同类型的3D内容的生成。 内容创建是具有挑战性的，因为高质量的3D形状通常是通过显式地对形状的几何形状建模来创建的。这一过程涉及熟练的艺术家在一个耗时的过程中使用复杂的用户界面，需要大量的培训。因此，在过去的二十年中，计算机图形学的研究也提出了方法，以促进非专业用户的三维形状建模，如参数模型和草图界面。然而，这些方法中的许多仍然需要用户具有足够的艺术技能，或者需要大量的手工工作来预处理数据。近年来，基于机器学习的3D内容合成方法引起了人们的极大兴趣，特别是基于深度神经网络（DNN）的方法，因为DNN提供了几个优于传统方法的优点，例如更容易的训练数据准备，无需手工制作特征提取方法，高泛化能力以及生成模型，允许用户合成类似于训练数据的新数据。然而，DNN目前具有某些限制，使其无法轻松用于形状建模。在这种情况下，我的研究计划的下一个资助期的目标是研究解决方案，以改善与DNN的3D形状的分析和合成。更详细地说：（1）我们建议开发方法，用于在使用DNN生成3D形状时实现更直接的用户控制，以便用户能够根据他们的目标设计形状;（2）我们建议使用与最新技术相比允许生成更高质量形状的表示，例如基于生成低复杂度可编辑形状的程序模型的表示;（3）我们建议使用程序模型来生成合成数据，这些数据可以用于训练形状分析DNN，而手动准备的数据较少，或者用于在受控环境中评估基于DNN的方法。所提出的工作的意义在于，所开发的解决方案将使指导合成高质量的形状与更少的手工工作。这将产生新的软件工具，这些工具将改善需要形状建模的行业的当前实践，使该技术的用户能够以更低的成本创建前所未有的多样性和大量内容。