CHS: Small: Collaborative Research: Sampling and Reconstruction for Computer Graphics Rendering and Imaging

高维信号的采样是图形绘制和计算摄影的核心，但遗憾的是，当前的方法仍然倾向于蛮力操作，需要大量样本，这既耗时又昂贵。在这个涉及两个机构的研究人员参与的项目中，首席调查员将在他们先前工作的基础上，为计算机图形绘制和成像中的采样和重建建立一个全面的理论、算法和系统基础。一个关键目标是一种统一的采样理论，该理论考虑视觉信号中的相干性类型(例如低等级、局部低等级、低频率、稀疏性)和测量类型(例如用于光传输采集的通用图案的渲染或投影中的点样本，或全光场图像的采集)。这将为选择最佳抽样策略和比较不同的方法提供一个统一的框架。它还将使建立严格的下界和最优结果成为可能。这项工作与信号处理、应用数学和摄影有直接的联系，并将在将这些领域与计算机图形学联系起来方面产生广泛的影响。首席调查员将部分通过将调查结果纳入其注册人数较多的在线课程来传播项目成果。他们还将提供数据集和软件，并将通过利用他们与多家高科技公司的紧密联系，努力将它们纳入工业应用。基于物理的渲染算法目前在生产中广泛使用，但照片级真实感渲染的效率仍然很低，因为它涉及到对每个像素的高维4D-8D蒙特卡罗积分的求值，考虑到抗锯齿、镜头效果、运动模糊、软阴影和全局照明。通常，每个像素都是单独处理的，每个整体维度都需要许多样本。类似的挑战出现在计算机图形学的其他领域，例如预计算渲染(4D-8D光传输操作员的显式制表)、光传输获取(像BRDF或BSSRDF这样的高维4D-8D功能的测量)，以及在消费型光场相机中获得更高维4D功能的计算摄影或成像。传统的方法是通过暴力(预先)计算或测量数据，然后进行压缩。然而，考虑到当前视觉外观数据集的大小和维度，这会产生不可接受的成本。在这项工作中，首席研究人员将利用连续(而不是离散傅立叶)域的稀疏性、光传输的相干性和结构来采样、重建和积分，减少数量级所需的数据量，同时为计算成像开发新的重建方案。在渲染方面，PI将探索一种新的方法，将运动模糊、景深和全局照明结合在一个基于不同效果的自适应蒙特卡罗采样和过滤的实时渲染算法中。这种方法的一个关键挑战是对困难路径进行稳健抽样；首席调查人员将通过保守的自适应抽样和分级大都会来解决这一问题。最后，将开发新的系统级软件，使渲染和成像的光传输模拟方法能够轻松集成和实施。

Sampling of high-dimensional signals is at the heart of graphical rendering and computational photography, but current approaches unfortunately still tend to be brute-force and require large numbers of samples, which is time-consuming and costly. In this project, which involves researchers at two institutions, the Principal Investigators will build on their prior work to develop a comprehensive theoretical, algorithmic and systems foundation for sampling and reconstruction in computer graphics rendering and imaging. A key goal is a unified sampling theory that considers the type of coherence in the visual signal (such as low rank, locally low rank, low frequency, sparsity) and the type of measurement (such as point samples in rendering or projection of generic patterns for light transport acquisition, or acquisition of full light field imagery). This will provide a unified framework for choosing the best sampling strategy, and for comparing different approaches. It will also enable the establishment of rigorous lower bounds and optimality results. The work has immediate connections to signal-processing, applied mathematics and photography, and will have broad impact in connecting these domains with computer graphics. The Principal Investigators will disseminate project outcomes in part by incorporating the findings into their online courses that have large enrolments. They will also make datasets and software available, and will work to include them in industrial applications by exploiting their strong ties with a number of high-tech companies. Physically-based rendering algorithms are now widespread in production, but photorealistic rendering is still inefficient since it involves the evaluation of a high-dimensional 4D-8D Monte Carlo integral for each pixel considering antialiasing, lens effects, motion blur, soft shadows and global illumination. Typically, each pixel is treated separately, with many samples needed for each integral dimension. Similar challenges arise in other areas of computer graphics, such as precomputed rendering (explicit tabulation of a 4D-8D light transport operator), light transport acquisition (measurement of high-dimensional 4D-8D functions like the BRDF or BSSRDF), and computational photography or imaging that acquires higher-dimensional 4D functions in consumer light field cameras. The traditional approach is to (pre)compute or measure the data by brute force, followed by compression. However, this incurs unacceptable costs given the size and dimensionality of current visual appearance datasets. In this work the Principal Investigators will leverage the sparsity in the continuous (rather than discrete Fourier) domain, coherence and structure of light transport to sample, reconstruct and integrate, reducing the amount of data needed by orders of magnitude, while developing new reconstruction schemes for computational imaging. Within rendering, the PIs will explore a novel method that combines motion blur, depth of field, and global illumination in a single algorithm for real-time rendering based on adaptive Monte Carlo sampling and filtering of different effects. A key challenge in such approaches is robust sampling of difficult paths; the Principal Investigators will address this issue with conservative adaptive sampling and Graduated Metropolis. Finally, new systems-level software will be developed that enables easy integration and implementation of light transport simulation methods for rendering and imaging.