CAREER: Simulation of Lighting and Acoustics in Interactive Virtual Environments

我的综合研究和教育计划的长期目标是开发交互式虚拟环境系统，支持包含多个交互用户的大型3D模型的逼真听觉/视觉模拟。研究计划拟议的研究项目的主要目标是开发有效的算法，通过散射在3D模型的表面来模拟光波和声波在虚拟环境中的传播。这一研究问题对于在动画虚拟环境中提供逼真的体验至关重要，因为它的解决方案使得全局照明和声音空间化成为可能。对于运动规划、逆向建模、场景捕获、热传递、无线电功率预测、火灾传播和交通分析等应用，它在沉浸式虚拟环境之外的其他领域也具有重要意义。对于照明和声学的模拟，基本问题是计算一个积分方程解，该积分方程用周围表面的波场表示每个点的波场。主要的困难是，由于遮挡、焦散和镜面高光，波场具有不连续性，这会在积分域的一小部分(即表面或方向)上导致较大的变化。以前基于辐射传递和蒙特卡罗路径跟踪的积分方法对于典型的虚拟环境通常不实用。我计划研究一种混合的光束跟踪和路径跟踪方法。通常的策略是跟踪光束，该光束将光线空间划分成与3D场景表面上的不同散射事件序列(混响序列)相对应的拓扑上不同的束，然后使用它们在交互式路径跟踪算法中指导采样。这种方法的动机是，在第一阶段跟踪的相对较少的波束的属性可以提供关于波场的有用信息，这些信息可以用于在第二阶段指导和加速对样本的评估。这种方法实现了以下有效的方法：1)混响序列的枚举，2)积分区域的分解，3)守恒逼近，4)射线交叉点的空间相干性，5)混响序列的采样，6)渐进细化，7)离线预计算。我们面临的挑战是开发一种方法来快速地通过3D模型跟踪光束，并在实际应用中获得被跟踪光束的好处。我的研究计划是研究混合光束跟踪和路径跟踪方法来解决虚拟环境中的经典问题。新的研究贡献将在四个层面上进行：理论、算法、应用和实验。首先，我计划研究新的理论，将波传播模拟为一组离散的混响路径，其中包括多种散射效应，包括绕射。其次，我计划开发新的算法，在一般3D模型中高效地找到具有一般类型散射的重要混响路径。第三，我计划研究所提出的计算混响路径的方法可以用于解决经典问题的新应用。最后，我计划进行实验，对计算模拟的结果进行定量和定性的评估，并与测量的波场进行比较。这项研究的总体成果将是一个计算框架和一套方法，用于计算一般3D模型中的一般混响路径，并在交互应用程序中评估它们。通过这个项目，我计划研究声音和光之间的协同，并将从一种波现象中学到的经验应用到另一种现象中。根据历史上的先例，我相信通过同时研究声和光，将有可能开发出更好的虚拟环境模拟。我的教育计划的目标是教学生并开发新的教育方法。特别是，我的一个特殊目标是使交互式虚拟环境系统能够在教育过程中使用。我计划通过设计新的跨学科课程来研究这种新的教育媒体，让来自不同背景的学生和教师学习和试验虚拟环境系统。我还计划开发新的教育材料(教科书、课程笔记、软件工具和3D模型)，指导学生(研究生、本科生和K-12)，并为弱势群体开发外展项目(指导贫困学生和部署残疾援助应用程序)。通过开发用于研究的交互式虚拟环境系统，以及调查它们在教学中的使用，我的研究和教育活动以独特的方式结合在一起。

The long-term goal of my integrated research and educationalplan is to develop interactive virtual environment sys-temssupporting realistic aural/visual simulation of large 3Dmodels containing multiple interacting users.Research PlanThe main objective of the proposed research project is todevelop efficient algorithms for simulating the propagationof light and sound waves through a virtual environment byscattering at surfaces of a 3D model. This research problemis central to providing a realistic experience in animmersive virtual environment, as its solution enablesglobal illumination and sound spatialization. It is alsosignificant for several other applications, including motionplanning, inverse modeling, scene capture, heat transfer,radio power prediction, fire propagation, and trafficanalysis, which are all important in fields beyond immersivevirtual environments.For simulations of both lighting and acoustics, thefundamental problem is to compute a solution to an integralequation expressing the wave field at every point in termsof the wave field on surrounding surfaces. The maindifficulty is that the wave field has discontinuities due toocclusions, caustics, and specular highlights, which inducelarge variations over small portions of the integrationdomain (i.e. surfaces or directions). Previous integrationmethods based on radiosity and Monte Carlo path tracing aregenerally not practical for typical virtual environments.I plan to investigate a hybrid beam tracing and path tracingapproach. The general strategy is to trace beams thatpartition the space of rays into topologically distinctbundles corresponding to different sequences of scatteringevents at surfaces of the 3D scene (reverberationsequences), and then use them to guide sampling in aninteractive path tracing algorithm. The motivation for thisapproach is that attributes of a relatively small number ofbeams traced during the first phase can provide usefulinformation about the wave field that can be used to guideand accelerate evaluation of samples during the secondphase. This approach enables efficient methods for: 1)enumeration of reverberation sequences, 2) decomposition ofintegration domain, 3) conservative approximation, 4)spatial coherence in ray intersections, 5) sampling ofreverberation sequences, 6) progressive refinement, and 7)off-line precomputation. The challenge is to develop methodsthat trace beams through 3D models quickly and reap thebenefits of the traced beams in useful applications.My research plan is to investigate hybrid beam tracing andpath tracing approaches to solve classical problems forvirtual environments. The new research contributions will bemade at four levels: theory, algorithms, applications, andexperiments. First, I plan to investigate new theory formodeling wave propagation as a discrete set of reverber-ation paths incorporating multiple scattering effectsincluding diffractions. Second, I plan to develop newalgorithms that efficiently find significant reverberationpaths with general types of scattering in general 3D models.Third, I plan to investigate new applications where theproposed approach for computing reverberation paths can beused to solve classical problems. Finally, I plan to performexperiments to evaluate the results of computed simulationsboth quantitatively and qualitatively in comparison tomeasured wave fields. The overall outcome of this researchwill be a computational framework and a suite of methods forcomputing general reverberation paths in general 3D modelsand evaluating them in interactive applications.Throughout this project, I plan to investigate the synergiesbetween sound and light and to apply the lessons learnedfrom one wave phenomenon to the other. Based on historicalprecedent, I believe that it will be possible to developbetter simulations of virtual environments by studying bothsound and light together.Educational PlanThe objectives of my educational plan are to teach studentsand to develop new methods for education. In particular, onespecial goal of mine is to enable the use of interactivevirtual environment systems in the educational process. Iplan to investigate this novel media for education bydesigning new interdisciplinary courses that will allowstudents and teachers from widely varying backgrounds tolearn about and experiment with virtual environment systems.I also plan to develop new educational materials (textbook,course notes, software tools, and 3D models), to mentorstudents (graduate, undergraduate, and K-12), and to developoutreach programs for disadvantaged people (mentoring under-privileged students and deploying handicap-assistingapplications).By both developing interactive virtual environment systemsfor research, and investigating their use for teaching, myresearch and educational activities are uniquely integrated.