CHS: Small: Collaborative Research: Sampling and Reconstruction for Computer Graphics Rendering and Imaging

高维信号的采样是图形渲染和计算摄影的核心，但不幸的是，目前的方法仍然倾向于暴力处理，需要大量的样本，这既耗时又昂贵。在这个涉及两个机构的研究人员的项目中，主要研究人员将在他们先前工作的基础上，为计算机图形渲染和成像的采样和重建开发一个全面的理论、算法和系统基础。一个关键的目标是一个统一的采样理论，该理论考虑了视觉信号的相干性类型（如低秩、局部低秩、低频、稀疏性）和测量类型（如用于光传输获取的通用模式的渲染或投影中的点样本，或全光场图像的获取）。这将为选择最佳抽样策略和比较不同的方法提供一个统一的框架。它还可以建立严格的下界和最优性结果。这项工作与信号处理、应用数学和摄影有直接的联系，并将在将这些领域与计算机图形学联系起来方面产生广泛的影响。主要研究人员将通过将研究结果纳入其大量注册的在线课程来部分传播项目成果。它们还将提供数据集和软件，并将利用它们与一些高科技公司的牢固关系，努力将它们纳入工业应用。基于物理的渲染算法现在在生产中广泛使用，但逼真的渲染仍然效率低下，因为它涉及到考虑抗锯齿、镜头效果、运动模糊、柔和阴影和全局照明的每个像素的高维4D-8D蒙特卡罗积分的评估。通常，每个像素被单独处理，每个积分维度需要许多样本。在计算机图形学的其他领域也出现了类似的挑战，例如预计算渲染（4D- 8d光传输算子的显式制表）、光传输获取（测量高维4D- 8d函数，如BRDF或BSSRDF），以及在消费光场相机中获取高维4D函数的计算摄影或成像。传统的方法是通过蛮力（brute force）（预）计算或测量数据，然后进行压缩。然而，考虑到当前视觉外观数据集的大小和维度，这会产生不可接受的成本。在这项工作中，主要研究人员将利用连续（而不是离散傅立叶）域的稀疏性、光传输的相干性和结构来采样、重建和整合，减少所需的数据量，同时为计算成像开发新的重建方案。在渲染中，pi将探索一种新的方法，该方法将运动模糊、景深和全局照明结合在一个基于自适应蒙特卡罗采样和不同效果过滤的实时渲染算法中。这些方法的一个关键挑战是对困难路径的鲁棒采样；主要研究人员将使用保守自适应采样和毕业大都市来解决这个问题。最后，将开发新的系统级软件，使渲染和成像的光传输模拟方法易于集成和实现。

Sampling of high-dimensional signals is at the heart of graphical rendering and computational photography, but current approaches unfortunately still tend to be brute-force and require large numbers of samples, which is time-consuming and costly. In this project, which involves researchers at two institutions, the Principal Investigators will build on their prior work to develop a comprehensive theoretical, algorithmic and systems foundation for sampling and reconstruction in computer graphics rendering and imaging. A key goal is a unified sampling theory that considers the type of coherence in the visual signal (such as low rank, locally low rank, low frequency, sparsity) and the type of measurement (such as point samples in rendering or projection of generic patterns for light transport acquisition, or acquisition of full light field imagery). This will provide a unified framework for choosing the best sampling strategy, and for comparing different approaches. It will also enable the establishment of rigorous lower bounds and optimality results. The work has immediate connections to signal-processing, applied mathematics and photography, and will have broad impact in connecting these domains with computer graphics. The Principal Investigators will disseminate project outcomes in part by incorporating the findings into their online courses that have large enrolments. They will also make datasets and software available, and will work to include them in industrial applications by exploiting their strong ties with a number of high-tech companies. Physically-based rendering algorithms are now widespread in production, but photorealistic rendering is still inefficient since it involves the evaluation of a high-dimensional 4D-8D Monte Carlo integral for each pixel considering antialiasing, lens effects, motion blur, soft shadows and global illumination. Typically, each pixel is treated separately, with many samples needed for each integral dimension. Similar challenges arise in other areas of computer graphics, such as precomputed rendering (explicit tabulation of a 4D-8D light transport operator), light transport acquisition (measurement of high-dimensional 4D-8D functions like the BRDF or BSSRDF), and computational photography or imaging that acquires higher-dimensional 4D functions in consumer light field cameras. The traditional approach is to (pre)compute or measure the data by brute force, followed by compression. However, this incurs unacceptable costs given the size and dimensionality of current visual appearance datasets. In this work the Principal Investigators will leverage the sparsity in the continuous (rather than discrete Fourier) domain, coherence and structure of light transport to sample, reconstruct and integrate, reducing the amount of data needed by orders of magnitude, while developing new reconstruction schemes for computational imaging. Within rendering, the PIs will explore a novel method that combines motion blur, depth of field, and global illumination in a single algorithm for real-time rendering based on adaptive Monte Carlo sampling and filtering of different effects. A key challenge in such approaches is robust sampling of difficult paths; the Principal Investigators will address this issue with conservative adaptive sampling and Graduated Metropolis. Finally, new systems-level software will be developed that enables easy integration and implementation of light transport simulation methods for rendering and imaging.