CAREER: Sparse Sampling and Reconstruction for Rendering Through Per-Scene Optimization

计算机图形渲染正日益成为我们日常生活中不可或缺的一部分，从产品和建筑设计到自动驾驶汽车的应用，仅举几个例子。在渲染环境中，通常使用蒙特卡罗(MC)算法通过随机采样生成图像，模拟从光源发射的光子如何与虚拟环境中的对象交互并到达摄影机。然而，准确重建图像需要模拟大量的光子，这使得这种方法在计算能力和能源消耗方面代价高昂。该项目将大大减少获得高质量图像所需的光子数量，从而显著降低成本。这一目标是通过寻找高似然光子路径的新方法以及通过增强渲染后的图像质量来实现的。项目成果将对上述应用程序以及图形以外也依赖MC集成的领域产生广泛影响。教育和推广活动将使用计算机图形学中引人注目的真实世界问题和概念，特别是渲染，作为激励工具，让不同年龄和人口背景的人对STEM感到兴奋。这项研究将专注于通过开发在渲染过程中或之后操作的新方法来加速MC渲染的融合。在此设置中使用的两种最常见的方法类别是重要性采样(期间)和重建/去噪(之后)。遗憾的是，现有的大多数重要抽样技术都是局部地提供路径指导，从而忽略了路径上未来点的模型误差。此外，当前最先进的重建/去噪方法使用在一组噪声图像及其对应的地面事实上训练的神经网络，并且这些方法在位于训练数据分布之外的测试图像上的性能可能是次优的。该项目通过引入以根本不同的方式看待这些问题的新颖框架来应对这些挑战。具体的研究目标有两个：(1)根据误差感知方式估计的分布来引导路径，这需要引入一个全局目标函数来考虑整个路径。(2)设计可适应手头测试实例的去噪系统，将去噪问题作为优化问题，并开发有效工作的新目标，而不需要地面真实图像。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Computer graphics rendering is increasingly becoming an integral part of our daily lives, in applications from product and architectural design to self-driving cars, to cite just a few examples. In the context of rendering, typically an image is generated through random sampling using what are known as Monte Carlo (MC) algorithms, simulating how the photons emitted from the light sources interact with the objects in the virtual environment and arrive at the camera. Accurate reconstruction of an image, however, requires simulating a large number of photons, making this approach costly in terms of computational power and energy consumption. This project will dramatically reduce the number of photons required to obtain a high-quality image, thereby significantly reducing the cost. This objective is achieved by novel ways of finding high-likelihood photon paths and by enhancing the quality of the image after rendering. Project outcomes will have broad impact on applications such as those mentioned above, as well as in areas beyond graphics that also rely on MC integration. Educational and outreach activities will use compelling real-world problems and concepts in computer graphics, particularly rendering, as motivational tools to get people of diverse ages and demographic backgrounds excited about STEM.This research will focus on accelerating the convergence of MC rendering by developing novel methods that operate during or after rendering. The two most common categories of methods used in this setting are importance sampling (during) and reconstruction/denoising (after). Unfortunately, the majority of existing importance sampling techniques provide path guidance locally and thereby ignore the model error on future points along the path. Moreover, current state-of-the-art reconstruction/denoising methods use a neural network trained on a set of noisy images and their corresponding ground truth, and the performance of these methods on test images that lie outside the distribution of the training data can be suboptimal. This project addresses these challenges by introducing novel frameworks that view these problems in a fundamentally different way. The specific research objectives are twofold: (1) To guide the paths according to distributions that are estimated in an error-aware manner, which requires introducing a global objective function that takes the entire path into consideration. (2) To design denoising systems that can be adapted to the test example at hand, by posing the denoising problem as an optimization problem and developing novel objectives that work effectively without the need for ground truth images.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.