CHS: Small: Predictive Material Appearance Modeling at Multiple Scales

沉浸式显示技术的最新进展，如虚拟现实(VR)耳机，使人们能够从第一人称的角度体验计算机模拟的虚拟世界。但要提供真正身临其境的虚拟体验，对丰富多样的材质外观进行预测性模拟至关重要。具体地说，因为一种材料的外观在不同的物理尺度上差异很大，所以需要新的方法来在非常不同的尺度上准确和一致地对其进行建模。这项研究的目标是开发新的技术，不仅能够在交互/VR应用中实现高保真的材质外观(例如，在飞行模拟中为用户提供对地形和天空的准确印象，或者在探索虚拟房间时提供各种装饰材料，如木材或金属)，而且还将有利于离线预测渲染任务。为了最大限度地扩大行业影响，PI将以开源形式发布整个软件体系结构，以便设计师、零售商、开发人员、教育工作者、艺术家和学生可以使用它。该项目的广泛影响将进一步加强，通过在研讨会和备受瞩目的会议教程，如SIGGRAPH/欧洲图形学课程，并通过利用新的外观建模技术来开发教学工具(例如，使用VR)延伸到高中生(特别是少数族裔)，以培养对STEM的兴趣。在计算机图形学和视觉中，已经开发了许多模型来计算描述和再现真实世界对象的外观。然而，这些模型通常是专门为在单个固定的物理范围内工作而设计的。例如，许多反射率模型将对象视为不透明的光滑表面；这些方法在远距离观察时效果很好，但在近距离观察时存在缺乏细微细节和不规则的问题。相比之下，微观外观模型通过高分辨率的体积或网格明确地捕捉到了材料的小尺度结构。由于它们的高度复杂性，这些模型最适合产生小对象的缩放视图，而表示大场景的成本可能高得令人望而却步。无法跨多个规模工作已成为构建高度沉浸式虚拟现实的主要障碍。这项研究的目标是开发新的计算工具，以在大范围内以一致和可预测的方式高效地模拟和再现材料外观。为此，将为数据驱动和离散随机模型开发新的外观建模技术，以及比例桥接算法，以确保模型在多个比例上高效和一致地工作。为了实现这一目标，将需要克服以下计算挑战：非线性-材料模型和最终外观之间的关系已知是高度非线性的，难以过滤和内插；非局部性复杂的光传输现象，如互反射，导致模型参数的局部材料变化影响全球材料外观；以及高昂的计算成本-在比例桥期间，通常需要在保持最终外观的同时搜索不同比例的模型参数，这通常需要解决昂贵的数值优化。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，认为值得支持。

Recent advances in immersive display technologies, such as virtual reality (VR) headsets, have allowed computer-simulated virtual worlds to be experienced from first-person perspectives. But to offer truly immersive virtual experiences, predictive simulation of richly diverse material appearance is crucial. Specifically, because a material's appearance varies greatly across different physical scales, new methods to model it accurately and consistently at greatly varying scales are needed. The goal of this research is to develop new techniques that will not only enable high-fidelity material appearance in interactive/VR applications (for example, providing the user with an accurate impression of the terrain and the sky in flight simulation, or of various decorative materials such as wood or metal when exploring virtual rooms), but will also benefit offline predictive rendering tasks. To maximize industrial impact, the PI will release the entire software architecture as open source so that it is available to designers, retailers, developers, educators, artists, and students. The project's broad impact will be further enhanced by presenting the findings in workshops and high-profile conference tutorials such as SIGGRAPH/Eurographics courses, and by leveraging the new appearance modeling techniques to develop pedagogical tools (e.g., using VR) for outreach to high-school students (especially minorities) to foster interest in STEM.In computer graphics and vision, many models have been developed to computationally describe and reproduce the appearance of real-world objects. However, these models are generally designed specifically to work at a single, fixed physical scale. Many reflectance models, for example, treat objects as opaque smooth surfaces; these methods work well when viewed from a distance, but suffer from a lack of fine-grained details and irregularities when viewed close-up. Micro-appearance models, in contrast, explicitly capture a material's small-scale structures via high-resolution volumes or meshes. Due to their high complexity, these models are best suited for producing zoomed views of small objects and can be prohibitively expensive to represent large scenes. The inability to work across multiple scales has become a major obstacle to building highly immersive virtual realities. The objective of this research is to develop new computational tools to model and reproduce material appearance efficiently in a consistent and predictive manner across greatly varying scales. To this end, new appearance modeling techniques will be developed for both data-driven and discrete stochastic models, along with scale-bridging algorithms to ensure that the models work efficiently and consistently across multiple scales. To achieve this goal, the following computational challenges will need to be overcome: Nonlinearity - the relation between material models and final appearance is known to be highly nonlinear and difficult to filter and interpolate; Non-locality - complex light transport phenomena such as inter-reflection cause local material changes of model parameters to affect material appearance globally; and High Computational Cost - during scale-bridging it is usually necessary to search for model parameters at different scales while preserving the final appearance, which generally requires solving expensive numerical optimizations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.