CAREER: Image-Based Methods and Technologies for Three-Dimensional Computer Graphics

基于图像的呈现（IBR）是计算机图形学的一种新方法，其中用于表示环境的底层模式由一组图像组成，而不是具有相关表面属性的几何原语的集合。与传统的计算机图形学方法相比，IBR提供了几个优点。首先，基于图像的表示允许用户以图形方式探索从现实世界捕获的对象和场景。其次，这些表现可以直接用现有的数码相机捕捉到，这种相机在未来会变得更加普遍。最后，IBR的显示算法需要适度的计算资源，使它们在台式计算机上实用。该研究计划将解决基于图像的渲染中的几个主要挑战。首先是开发新算法，以克服现有基于图像的渲染技术的局限性，包括：减少这些算法对对应信息的依赖，扩展IBR技术以适应动态环境，设计支持沉浸式场景表示的技术，以及设计紧凑的视图依赖表示。将解决的第二组挑战是基于图像的场景表示的自动获取。这将通过将随机访问的图像传感器阵列与生成所需渲染所需的计算元素紧密耦合来完成。这种方法的好处是，它可以在架构上处理实时场景，同时最佳地利用可用的通信带宽并执行最小的必要计算。本研究解决的第三个挑战是对高性能图像渲染硬件的需求。计算机图形学的历史告诉我们，将系统交到早期应用程序开发人员手中对该领域的发展至关重要，因为它提供了重要的反馈，形成了未来的研究议程。高性能硬件是向应用程序开发人员提供IBR技术的关键一步。这项研究的结果将有利于许多不同的领域，如深水或行星际探索的远程呈现和医疗应用。这项研究还将为电影行业的特效、视觉影响研究、建筑可视化、军事模拟器和视频游戏提供新一代的工具。

Image-based rendering (IBR) is a new approach to computer graphics in which the underlying mode used to represent an environment is composed of a set of images rather than a collection of geometric primitives with associated surface properties. IBR offers several advantages over traditional computer graphics approaches. First, image-based representations allow users to graphically explore objects and scenes that are captured from the real world. Second, these representations may be captured directly using digital cameras that exist today and which will become more prevalent in the future. Finally, the display algorithms for IBR require modest computational resources, making them practical on desktop computers.This research plan will address several major challenges in image-based rendering. The first is the development of new algorithms to overcome the limitations of existing image-based rendering techniques including: reducing the dependence of these algorithms on correspondence information, extending IBR techniques to accommodate dynamic environments, devising techniques to support immersive scene representations, and devising compact view-dependent representations. A second set of challenges that will be addressed is the automatic acquisition of image-based scene representations. This will be done by tightly coupling an array of randomly accessed image sensors with the computational elements required to generate the desired rendering. The benefit of this approach is that it can be architected to handle real-time scenes while making optimal use of available communication bandwidth and performing the minimum computation necessary. The third challenge addressed by this research is the need for high-performance image-based rendering hardware. The history of computer graphics has taught us that getting systems into the hands of early application developers is crucial to the field's development because it provides important feedback that shapes the future research agenda. High-performance hardware is a key step toward placing IBR techniques into the hands of application developers.The results of this research will benefit many different areas such as remote telepresence for deep-water or interplanetary exploration and medical applications. The research will also provide a new generation of tools for special effects in the motion picture industry, visual impact studies, architectural visualizations, military simulators, and video games.