Robust flow-guided neural prediction for sketch-based freeform surface modeling
Robust flow-guided neural prediction for sketch-based freeform surface modeling
复制标题
DOI：
10.1145/3272127.3275051
复制
发表时间：
2018-12
期刊：
影响因子：
--
通讯作者：
Changjian Li;Hao Pan;Yang Liu;Xin Tong;A. Sheffer;Wenping Wang
中科院分区：
文献类型：
--
作者：
Changjian Li;Hao Pan;Yang Liu;Xin Tong;A. Sheffer;Wenping Wang
素描提供了一个直观的用户界面，用于交流自由形状。虽然人类观察者可以很容易地想象出他们想要交流的形状，但复制这一过程需要通过算法解决许多歧义。现有的基于草图的建模方法通过依赖昂贵的用户注释或将建模的形状限制在特定的狭窄类别来解决这些模糊性。我们提出了一种从稀疏的、富有表现力的2D草图中建模通用自由曲面3D表面的方法，该方法通过将卷积神经网络（CNN）纳入草图处理工作流程来克服这两种限制。给定一个3D表面的2D草图，我们使用cnn来推断表示该表面的深度和法线贴图。为了消除歧义，我们引入了一个中间的CNN层，该层对表面的密集曲率方向或流场进行建模，并产生一个附加的输出置信图，以及深度和法线。流场指导我们后续的表面重建，以改善规律性；该置信图训练了无监督度量的模糊性，并为数据拟合提供了一个稳健的估计器。为了减少输入草图中的歧义，用户可以通过在稀疏点提供可选的深度值和笔画的曲率提示来细化输入。我们的CNN是在一个大型数据集上训练的，该数据集是通过使用非照片真实感线渲染（NPR）方法绘制各种3D形状的草图而生成的，该方法模仿人类绘制自由形状的草图。我们使用CNN模型来处理单视图和多视图草图。使用我们的多视图框架，用户通过在不同的视图中绘制草图来逐步完成形状，生成完整的封闭形状。对于每个新视图，建模是由部分草图和深度线索提供的表面生成的早期视图。部分表面被融合成一个完整的形状，使用预测的置信度作为权重。我们验证了我们的方法，将其与以前的方法和替代结构进行比较，并在各种建模任务中评估其性能。结果表明，我们的方法是一种新的方法，可以用简洁而富有表现力的2D草图高效地建模自由形状。
Sketching provides an intuitive user interface for communicating free form shapes. While human observers can easily envision the shapes they intend to communicate, replicating this process algorithmically requires resolving numerous ambiguities. Existing sketch-based modeling methods resolve these ambiguities by either relying on expensive user annotations or by restricting the modeled shapes to specific narrow categories. We present an approach for modeling generic freeform 3D surfaces from sparse, expressive 2D sketches that overcomes both limitations by incorporating convolution neural networks (CNN) into the sketch processing workflow. Given a 2D sketch of a 3D surface, we use CNNs to infer the depth and normal maps representing the surface. To combat ambiguity we introduce an intermediate CNN layer that models the dense curvature direction, or flow, field of the surface, and produce an additional output confidence map along with depth and normal. The flow field guides our subsequent surface reconstruction for improved regularity; the confidence map trained unsupervised measures ambiguity and provides a robust estimator for data fitting. To reduce ambiguities in input sketches users can refine their input by providing optional depth values at sparse points and curvature hints for strokes. Our CNN is trained on a large dataset generated by rendering sketches of various 3D shapes using non-photo-realistic line rendering (NPR) method that mimics human sketching of free-form shapes. We use the CNN model to process both single- and multi-view sketches. Using our multi-view framework users progressively complete the shape by sketching in different views, generating complete closed shapes. For each new view, the modeling is assisted by partial sketches and depth cues provided by surfaces generated in earlier views. The partial surfaces are fused into a complete shape using predicted confidence levels as weights. We validate our approach, compare it with previous methods and alternative structures, and evaluate its performance with various modeling tasks. The results demonstrate our method is a new approach for efficiently modeling freeform shapes with succinct but expressive 2D sketches.