CAREER: Sparse Sampling and Reconstruction for Rendering Through Per-Scene Optimization

计算机图形渲染正日益成为我们日常生活中不可或缺的一部分，从产品和建筑设计到自动驾驶汽车，仅举几个例子。在渲染的上下文中，通常使用所谓的蒙特卡罗（MC）算法通过随机采样生成图像，模拟从光源发射的光子如何与虚拟环境中的对象交互并到达相机。然而，图像的精确重建需要模拟大量的光子，使得这种方法在计算能力和能耗方面成本高昂。该项目将大大减少获得高质量图像所需的光子数量，从而大大降低成本。这一目标是通过寻找高似然光子路径的新方法和通过增强渲染后图像的质量来实现的。项目成果将对上述应用程序以及依赖MC集成的图形以外的领域产生广泛影响。教育和推广活动将使用计算机图形学中引人注目的现实世界问题和概念，特别是渲染，作为激励工具，让不同年龄和人口背景的人对STEM感到兴奋。这项研究将专注于通过开发在渲染期间或之后操作的新方法来加速MC渲染的融合。在此设置中使用的两种最常见的方法类别是重要性采样（期间）和重建/去噪（之后）。不幸的是，大多数现有的重要性采样技术提供的路径指导本地，从而忽略了模型误差的未来点沿着path. Further，目前的国家的最先进的重建/降噪方法使用的神经网络上训练的一组噪声图像和它们相应的地面真理，这些方法的性能测试图像，位于分布的训练数据之外，可以是次优的。该项目通过引入新颖的框架来解决这些挑战，这些框架以一种根本不同的方式看待这些问题。具体的研究目标有两个方面：（1）根据以误差感知的方式估计的分布来引导路径，这需要引入考虑整个路径的全局目标函数。(2)通过将去噪问题视为优化问题，并开发无需地面实况图像即可有效工作的新目标，设计可适应当前测试示例的去噪系统。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Computer graphics rendering is increasingly becoming an integral part of our daily lives, in applications from product and architectural design to self-driving cars, to cite just a few examples. In the context of rendering, typically an image is generated through random sampling using what are known as Monte Carlo (MC) algorithms, simulating how the photons emitted from the light sources interact with the objects in the virtual environment and arrive at the camera. Accurate reconstruction of an image, however, requires simulating a large number of photons, making this approach costly in terms of computational power and energy consumption. This project will dramatically reduce the number of photons required to obtain a high-quality image, thereby significantly reducing the cost. This objective is achieved by novel ways of finding high-likelihood photon paths and by enhancing the quality of the image after rendering. Project outcomes will have broad impact on applications such as those mentioned above, as well as in areas beyond graphics that also rely on MC integration. Educational and outreach activities will use compelling real-world problems and concepts in computer graphics, particularly rendering, as motivational tools to get people of diverse ages and demographic backgrounds excited about STEM.This research will focus on accelerating the convergence of MC rendering by developing novel methods that operate during or after rendering. The two most common categories of methods used in this setting are importance sampling (during) and reconstruction/denoising (after). Unfortunately, the majority of existing importance sampling techniques provide path guidance locally and thereby ignore the model error on future points along the path. Moreover, current state-of-the-art reconstruction/denoising methods use a neural network trained on a set of noisy images and their corresponding ground truth, and the performance of these methods on test images that lie outside the distribution of the training data can be suboptimal. This project addresses these challenges by introducing novel frameworks that view these problems in a fundamentally different way. The specific research objectives are twofold: (1) To guide the paths according to distributions that are estimated in an error-aware manner, which requires introducing a global objective function that takes the entire path into consideration. (2) To design denoising systems that can be adapted to the test example at hand, by posing the denoising problem as an optimization problem and developing novel objectives that work effectively without the need for ground truth images.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.