CHS: Small: Predictive Material Appearance Modeling at Multiple Scales

沉浸式显示技术的最新进展，如虚拟现实（VR）耳机，已经允许从第一人称视角体验计算机模拟的虚拟世界。但要提供真正的沉浸式虚拟体验，对丰富多样的材料外观进行预测模拟至关重要。具体来说，由于材料的外观在不同的物理尺度上变化很大，因此需要在不同的物理尺度上精确和一致地对其进行建模的新方法。这项研究的目标是开发新技术，不仅可以在交互式/虚拟现实应用中实现高保真的材料外观（例如，在飞行模拟中为用户提供地形和天空的准确印象，或者在探索虚拟房间时为用户提供各种装饰材料（如木材或金属）），而且还将有利于离线预测渲染任务。为了最大限度地发挥工业影响，PI将把整个软件架构作为开源发布，以便设计师、零售商、开发人员、教育工作者、艺术家和学生都可以使用它。通过在研讨会和高知名度的会议教程（如SIGGRAPH/Eurographics课程）中展示研究结果，以及利用新的外观建模技术开发教学工具（例如，使用VR），以促进高中生（特别是少数民族）对STEM的兴趣，该项目的广泛影响将进一步增强。在计算机图形学和视觉中，已经开发了许多模型来计算地描述和再现现实世界物体的外观。然而，这些模型通常是专门设计在一个单一的，固定的物理规模。例如，许多反射模型将物体视为不透明的光滑表面；这些方法在从远处观察时效果很好，但在近距离观察时缺乏细粒度的细节和不规则性。相比之下，微观外观模型通过高分辨率的体积或网格明确地捕捉材料的小规模结构。由于它们的高复杂性，这些模型最适合于生成小对象的缩放视图，并且对于表示大场景来说可能过于昂贵。无法跨多个尺度工作已经成为构建高度沉浸式虚拟现实的主要障碍。本研究的目的是开发新的计算工具，以一致和预测的方式在不同的尺度上有效地模拟和再现材料的外观。为此，将为数据驱动和离散随机模型开发新的外观建模技术，以及尺度桥接算法，以确保模型在多个尺度上有效和一致地工作。为了实现这一目标，需要克服以下计算挑战：非线性-材料模型和最终外观之间的关系已知是高度非线性的，难以过滤和插值；非局域复杂的光输运现象，如间反射，会引起模型参数的局部变化，从而影响材料的整体外观；计算成本高-在尺度桥接过程中，通常需要在保留最终外观的同时搜索不同尺度的模型参数，这通常需要解决昂贵的数值优化问题。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recent advances in immersive display technologies, such as virtual reality (VR) headsets, have allowed computer-simulated virtual worlds to be experienced from first-person perspectives. But to offer truly immersive virtual experiences, predictive simulation of richly diverse material appearance is crucial. Specifically, because a material's appearance varies greatly across different physical scales, new methods to model it accurately and consistently at greatly varying scales are needed. The goal of this research is to develop new techniques that will not only enable high-fidelity material appearance in interactive/VR applications (for example, providing the user with an accurate impression of the terrain and the sky in flight simulation, or of various decorative materials such as wood or metal when exploring virtual rooms), but will also benefit offline predictive rendering tasks. To maximize industrial impact, the PI will release the entire software architecture as open source so that it is available to designers, retailers, developers, educators, artists, and students. The project's broad impact will be further enhanced by presenting the findings in workshops and high-profile conference tutorials such as SIGGRAPH/Eurographics courses, and by leveraging the new appearance modeling techniques to develop pedagogical tools (e.g., using VR) for outreach to high-school students (especially minorities) to foster interest in STEM.In computer graphics and vision, many models have been developed to computationally describe and reproduce the appearance of real-world objects. However, these models are generally designed specifically to work at a single, fixed physical scale. Many reflectance models, for example, treat objects as opaque smooth surfaces; these methods work well when viewed from a distance, but suffer from a lack of fine-grained details and irregularities when viewed close-up. Micro-appearance models, in contrast, explicitly capture a material's small-scale structures via high-resolution volumes or meshes. Due to their high complexity, these models are best suited for producing zoomed views of small objects and can be prohibitively expensive to represent large scenes. The inability to work across multiple scales has become a major obstacle to building highly immersive virtual realities. The objective of this research is to develop new computational tools to model and reproduce material appearance efficiently in a consistent and predictive manner across greatly varying scales. To this end, new appearance modeling techniques will be developed for both data-driven and discrete stochastic models, along with scale-bridging algorithms to ensure that the models work efficiently and consistently across multiple scales. To achieve this goal, the following computational challenges will need to be overcome: Nonlinearity - the relation between material models and final appearance is known to be highly nonlinear and difficult to filter and interpolate; Non-locality - complex light transport phenomena such as inter-reflection cause local material changes of model parameters to affect material appearance globally; and High Computational Cost - during scale-bridging it is usually necessary to search for model parameters at different scales while preserving the final appearance, which generally requires solving expensive numerical optimizations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.