CAREER: Automatic and general light transport algorithms for use by humans and machines

计算机图形渲染使用光传输模拟来虚拟地“捕捉”照片。虽然它的发展在很大程度上反映了摄影的发展，但不幸的是，今天的图形状态就像20世纪中期的摄影一样，当时创建一幅好的图像需要一个训练有素的专业人员，对光和可用工具的特性有深入的了解。从那时起，摄影见证了一场革命，智能手机相机在隐藏这种复杂性的同时，产生了技术上优越的图像；这种民主化使大量新的应用成为可能。当前项目的目标是为使用计算机图形算法创建的“虚拟照片”实现同样的革命。这将需要开发表达数字光传输模拟的新方法，该方法放松当前对渲染过程的输入和输出的假设，从而产生避免手动参数调整的自动“点击”渲染算法。项目成果将对目前依赖准确和高效的光传输模拟的所有应用领域产生深远和广泛的影响，从诊断疾病到自动驾驶汽车，再到视频游戏和电影，再到设计和制造产品。更重要的是，他们将使这些工具大众化，超越训练有素的专业人士，就像智能手机相机为摄影所做的那样。一个综合的教育和推广计划将增加多样性，并通过建立主办机构在STEM和数字艺术方面的现有优势来支持未被充分代表的群体，并将利用动画电影的吸引力向来自广泛年龄和社会经济背景的学生教授原本抽象的数学和物理概念。虽然计算机图形渲染可以产生我们今天在电影中看到的逼真而美丽的图像，但用于实现这些结果的算法是脆弱的，需要熟练的从业者进行调整。它们几乎完全专注于模仿传统的相机/光学系统，这些系统是为人们设计的，这限制了它们对新兴成像方式的适用性。由于对物理世界和其中的光传输的不切实际的假设，他们仍然难以渲染常见的场景。要扩大计算机图形学的应用范围，需要解决这些核心挑战。首先，这个项目将建立一个新的框架来表达蒙特卡罗光传输策略，并建立一种计算方法来确定哪些场景应该使用哪些策略。这将导致更健壮和自动的“点击式”渲染算法，避免使用启发式手动调整参数。这项研究还将把目前的渲染方法扩展到2D RGB图像之外，以使用现代成像系统对反问题产生更广泛的影响。最后，这项工作将解除长期持有的关于光传输和场景/材质表示的统计假设，为普遍的细节水平和增加艺术控制奠定理论和算法基础。项目成果将对依赖计算机图形绘制的所有应用领域产生深远影响，并将增加有关核工程和大气科学等相关领域的非经典运输的宝贵基础知识。更广泛地说，这项工作将把未来渲染研究的重点转移到稳健和自动渲染算法上，以使这些工具在计算机图形专业人员之外大众化。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Computer graphics rendering uses light transport simulation to virtually "capture" photographs. While its evolution has largely mirrored that of photography, the state of graphics today is unfortunately like that of photography in the mid-1900s when creating a good image required being a trained professional with an intimate understanding of light and the peculiarities of the available tools. Since then, photography has witnessed a revolution where smartphone cameras produce technically superior images while hiding this complexity; this democratization has enabled a vast array of new applications. The goal of the current project is to enable the same revolution for the "virtual photographs" created using computer graphics algorithms. This will require developing new ways of expressing numerical light transport simulations which relax current assumptions on the input and output of the rendering process, leading to automatic "point and click" rendering algorithms which avoid manual parameter tuning. Project outcomes will have far-reaching and broad impact on all application areas that currently rely on accurate and efficient light transport simulation, from diagnosing illnesses, to self-driving cars, to video games and films, to designing and manufacturing products. More importantly, they will democratize these tools beyond trained professionals, much like "point and shoot" smartphone cameras have done for photography. An integrated educational and outreach program will increase diversity and support underrepresented groups by building on the host institution's existing strengths in STEM and Digital Arts, and will leverage the appeal of animated films to teach otherwise abstract mathematics and physics concepts to students from a broad range of ages and socio-economic backgrounds.While computer graphics rendering can produce the realistic and beautiful imagery we see in movies today, the algorithms used to achieve these results are brittle and require tuning by skilled practitioners. They focus almost entirely on mimicking traditional camera/optical systems that were designed for use by people, which limits their applicability to emerging imaging modalities. And they still struggle to render common scenes due to unrealistic assumptions about the physical world and the light transport within it. Broadening the reach of computer graphics requires addressing these core challenges. Firstly, this project will establish a new framework for expressing Monte Carlo light transport strategies and a method to computationally determine which ones should be used on which scenes. This will lead to more robust and automatic "point and click" rendering algorithms which avoid manual parameter tuning with heuristics. The research will also extend current rendering approaches beyond 2D RGB images for broader impact on inverse problems using modern imaging systems. Lastly, the work will lift long-held statistical assumptions about light transport and scene/material representations, laying the theoretical and algorithmic foundation for universal level of detail and increased artistic control. Project outcomes will have far-reaching impact in all application domains relying on computer graphics rendering, and will add valuable basic knowledge about non-classical transport to related domains like nuclear engineering and atmospheric science. More broadly, the work will shift the focus of future rendering research towards robust and automatic rendering algorithms to democratize these tools beyond computer graphics professionals.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.