CAREER: Image-Based Methods and Technologies for Three-Dimensional Computer Graphics

基于图像的绘制(IBR)是一种新的计算机图形学方法，其中用于表示环境的底层模式由一组图像组成，而不是具有相关表面属性的几何基元的集合。与传统的计算机图形方法相比，IBR有几个优势。首先，基于图像的表示允许用户以图形方式探索从真实世界捕获的对象和场景。其次，这些表现形式可以直接使用今天存在的并将在未来变得更加普遍的数码相机来捕捉。最后，IBR的显示算法需要适度的计算资源，使其在台式计算机上实用。该研究计划将解决基于图像绘制的几个主要挑战。首先是开发新的算法来克服现有基于图像的绘制技术的局限性，包括：减少这些算法对对应信息的依赖，扩展IBR技术以适应动态环境，设计支持沉浸式场景表示的技术，以及设计紧凑的依赖于视图的表示。将解决的第二组挑战是基于图像的场景表示的自动获取。这将通过将随机访问的图像传感器阵列与生成所需渲染所需的计算元素紧密耦合来实现。这种方法的好处是，它可以被设计为处理实时场景，同时优化利用可用通信带宽并执行必要的最小计算。这项研究解决的第三个挑战是对高性能基于图像的绘制硬件的需求。计算机图形学的历史告诉我们，让早期应用程序开发人员掌握系统对该领域的发展至关重要，因为它提供了重要的反馈，塑造了未来的研究议程。高性能硬件是将IBR技术应用到应用程序开发人员手中的关键一步。这项研究的结果将使许多不同领域受益，如用于深水或星际探测的远程远程临场感和医疗应用。这项研究还将为电影工业、视觉影响研究、建筑可视化、军事模拟器和视频游戏中的特殊效果提供新一代工具。

Image-based rendering (IBR) is a new approach to computer graphics in which the underlying mode used to represent an environment is composed of a set of images rather than a collection of geometric primitives with associated surface properties. IBR offers several advantages over traditional computer graphics approaches. First, image-based representations allow users to graphically explore objects and scenes that are captured from the real world. Second, these representations may be captured directly using digital cameras that exist today and which will become more prevalent in the future. Finally, the display algorithms for IBR require modest computational resources, making them practical on desktop computers.This research plan will address several major challenges in image-based rendering. The first is the development of new algorithms to overcome the limitations of existing image-based rendering techniques including: reducing the dependence of these algorithms on correspondence information, extending IBR techniques to accommodate dynamic environments, devising techniques to support immersive scene representations, and devising compact view-dependent representations. A second set of challenges that will be addressed is the automatic acquisition of image-based scene representations. This will be done by tightly coupling an array of randomly accessed image sensors with the computational elements required to generate the desired rendering. The benefit of this approach is that it can be architected to handle real-time scenes while making optimal use of available communication bandwidth and performing the minimum computation necessary. The third challenge addressed by this research is the need for high-performance image-based rendering hardware. The history of computer graphics has taught us that getting systems into the hands of early application developers is crucial to the field's development because it provides important feedback that shapes the future research agenda. High-performance hardware is a key step toward placing IBR techniques into the hands of application developers.The results of this research will benefit many different areas such as remote telepresence for deep-water or interplanetary exploration and medical applications. The research will also provide a new generation of tools for special effects in the motion picture industry, visual impact studies, architectural visualizations, military simulators, and video games.