CAREER: Sparse Sampling and Reconstruction for Rendering Through Per-Scene Optimization

计算机图形渲染正日益成为我们日常生活中不可或缺的一部分，从产品和建筑设计到自动驾驶汽车的应用，仅举几个例子。在渲染的背景下，通常图像是通过随机抽样生成的，使用所谓的蒙特卡罗（MC）算法，模拟光源发射的光子如何与虚拟环境中的物体相互作用并到达相机。然而，精确的图像重建需要模拟大量的光子，这使得这种方法在计算能力和能量消耗方面代价高昂。该项目将显著减少获得高质量图像所需的光子数量，从而显著降低成本。这一目标是通过寻找高似然光子路径的新方法和通过增强渲染后的图像质量来实现的。项目成果将对上面提到的应用产生广泛的影响，也会对依赖MC集成的图形以外的领域产生广泛的影响。教育和推广活动将使用计算机图形学中引人注目的现实问题和概念，特别是渲染，作为激励工具，让不同年龄和人口背景的人们对STEM感到兴奋。本研究将通过开发在渲染过程中或之后操作的新方法来加速MC渲染的融合。在这种情况下使用的两种最常见的方法是重要性采样（过程）和重建/去噪（之后）。不幸的是，大多数现有的重要性采样技术提供了局部路径引导，从而忽略了路径上未来点的模型误差。此外，目前最先进的重建/去噪方法使用的是在一组噪声图像及其相应的地面真值上训练的神经网络，这些方法在训练数据分布之外的测试图像上的性能可能不是最优的。本项目通过引入新颖的框架来解决这些挑战，这些框架以一种根本不同的方式看待这些问题。具体的研究目标有两个：(1)根据误差感知方式估计的分布来引导路径，这需要引入一个考虑整个路径的全局目标函数。(2)通过将去噪问题作为优化问题并开发无需地面真值图像即可有效工作的新目标，设计可适应手头测试示例的去噪系统。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Computer graphics rendering is increasingly becoming an integral part of our daily lives, in applications from product and architectural design to self-driving cars, to cite just a few examples. In the context of rendering, typically an image is generated through random sampling using what are known as Monte Carlo (MC) algorithms, simulating how the photons emitted from the light sources interact with the objects in the virtual environment and arrive at the camera. Accurate reconstruction of an image, however, requires simulating a large number of photons, making this approach costly in terms of computational power and energy consumption. This project will dramatically reduce the number of photons required to obtain a high-quality image, thereby significantly reducing the cost. This objective is achieved by novel ways of finding high-likelihood photon paths and by enhancing the quality of the image after rendering. Project outcomes will have broad impact on applications such as those mentioned above, as well as in areas beyond graphics that also rely on MC integration. Educational and outreach activities will use compelling real-world problems and concepts in computer graphics, particularly rendering, as motivational tools to get people of diverse ages and demographic backgrounds excited about STEM.This research will focus on accelerating the convergence of MC rendering by developing novel methods that operate during or after rendering. The two most common categories of methods used in this setting are importance sampling (during) and reconstruction/denoising (after). Unfortunately, the majority of existing importance sampling techniques provide path guidance locally and thereby ignore the model error on future points along the path. Moreover, current state-of-the-art reconstruction/denoising methods use a neural network trained on a set of noisy images and their corresponding ground truth, and the performance of these methods on test images that lie outside the distribution of the training data can be suboptimal. This project addresses these challenges by introducing novel frameworks that view these problems in a fundamentally different way. The specific research objectives are twofold: (1) To guide the paths according to distributions that are estimated in an error-aware manner, which requires introducing a global objective function that takes the entire path into consideration. (2) To design denoising systems that can be adapted to the test example at hand, by posing the denoising problem as an optimization problem and developing novel objectives that work effectively without the need for ground truth images.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.