Images with Normals: Acquisition, Analysis, and Depiction

摘要：基于计算机的3D模型程式化描述方法的发展有望帮助科学家和工程师制作清晰而引人注目的插图和可视化。然而，尽管在使3D采集变得廉价和实用方面取得了稳步进展，但获得复杂物体的完整3D模型仍然具有挑战性。该项目研究了从简单的2D图像和完整的3D模型之间的数据类型创建插图：每个像素存储一个表面法线的图像。这些“RGBN图像”具有成为广泛使用的数据类型的潜力，因为它们可以轻松，灵活和高质量地获得，并且因为它们包含足够的信息来允许许多分析和描述任务。也就是说，它们将采集过程与最初为全3D模型开发的工具的功能和灵活性结合起来，只比数码照片稍微复杂一点。RGBN形状分析和非真实感渲染方法将允许在医学和技术插图、艺术史和法医分析等领域探索和交流表面形状和细节。该项目包括对RGBN图像数据类型的全面调查，目的是开发一种实用的管道，用于获取具有法线的图像并生成风格化描述。在采集方面，该项目正在开发硬件/软件采集系统，用于在从毫米级物体到城市景观的环境中可靠地获取RGBN图像，包括静态和移动物体。接下来，该项目包括对RGBN图像信号处理方法的数学分析，包括尺度空间分析和导数估计。这些信号处理技术用于开发描绘形状和颜色的方法，包括阴影，带有暗示性轮廓和折痕线的线条绘制，夸张的阴影以及深度不连续的增强。最后，正在开发RGBN分析和处理算法，如纹理分析/合成，着色和基于相似性的搜索。

RusinkiewiczPrinceton UniversityAbstract:The development of computer-based methods for stylized depiction of 3D models promises to help scientists and engineers produce clear and compelling illustrations and visualizations. However, despite steady progress on making 3D acquisition inexpensive and practical, obtaining complete 3D models of complex objects remains challenging. This project investigates the creation of illustrations from a data type lying between simple 2D images and full 3D models: images with a surface normal stored at each pixel. These ``RGBN images'' have the potential of becoming a widely-used data type because of the ease, flexibility, and quality with which they may be acquired, and because they contain enough information to permit many analysis and depiction tasks. That is, they combine an acquisition process only mildly more complex than that for digital photographs with the power and flexibility of tools originally developed for full 3D models. Methods for RGBN shape analysis and nonphotorealistic rendering will allow for exploration and communication of surface shape and detail in domains such as medical and technical illustration, art history, and forensic analysis.This project encompasses a comprehensive investigation of the RGBN image data type, with the aim of developing a practical pipeline for acquiring images with normals and generating stylized depictions. On the acquisition side, the project is developing hardware/software acquisition systems for robustly acquiring RGBN images in contexts ranging from millimeter-scale objects through cityscapes, and including both static and moving objects. Next, the project includes a mathematical analysis of methods for signal processing on RGBN images, including scale-space analysis and derivative estimation. These signal processing techniques are used to develop methods for depicting shape and color, including shading, line drawing with suggestive contours and crease lines, exaggerated shading, and enhancement of depth discontinuities. Finally, RGBN analysis and processing algorithms such as texture analysis/synthesis, inpainting, and similarity-based search are being developed.