CIF: Medium: Mobile multiview video: compression, rendering, and transmission

3D视频技术突飞猛进。2009年的3D电影《阿凡达》成为有史以来票房最高的电影。这样的电影需要一个场景的左右视图。许多提供3D体验的电子游戏需要多个场景视图。这种数据的存储和传输成本很高。本课题研究了如何高效压缩多视点视频数据，如何从任意角度以不同精度观看场景，以及如何通过移动无线信道可靠地传输数据。该研究在科学教育、交通监控、监控安防等领域具有重要的应用价值。本研究研究多视点视频的高效编码、渲染和传输，旨在实现移动设备在任意速度下的鲁棒性能。该研究既适用于多摄像机拍摄的真实世界视频，也适用于渲染视频。研究者研究了H.264 MVC标准中的左/右视图编码，以及视图+深度编码。后一种方法通过对原始视图与其解码器合成版本之间的错误信号进行编码来增强。为了优化设计系统，该技术使用了跨层优化，其中利用物理层信道状态信息和应用层失真率或切片优先级信息。每当从底层3D虚拟世界中渲染多个视图时，应用程序的位需求可能会因渲染参数而发生巨大变化，这些参数会影响场景的内容和细节水平。研究人员研究了用户体验模型，以量化渲染参数与用户满意度之间的关系，并开发了一种信道感知自适应编码和渲染算法，以解释移动信道上传输引起的波动。

The prominence of 3D video technology has skyrocketed. The 2009 movie Avatar in 3D became the highest grossing movie of all time. Such a movie requires left and right views of a scene. Many video games which provide a 3D experience require multiple views of a scene. Such data is costly to store and to transmit.This research studies how to efficiently compress multiple-view video data, how to allow the scene to be viewed from any angle at different levels of precision, and how to reliably transmit the data over mobile wireless channels. This research has important applications in science education, traffic monitoring, and surveillance and security.This research studies efficient encoding, rendering, and transmission of multi-view video, aiming for robust performance at arbitrary speeds of mobile units. The research is applicable both to videos of the real world taken with multiple cameras, and to rendered videos. The investigators study left/right view coding such as in the H.264 MVC standard, and view+depth coding. The latter approach is enhanced by encoding the error signal between the original view and its decoder-synthesized version. To optimally design the system, the techniques use cross-layer optimization, in which physical-layer channel-state information and application-layer distortion-rate or slice-priority information are exploited. Whenever multiple views are rendered from an underlying 3D virtual world, the application's bit requirements can be hugely altered by rendering parameters which affect the content and level of detail of the scene. The investigators study user-experience models to quantify the relationship between rendering parameters and user satisfaction, and develop a channel-aware adaptive encoding and rendering algorithm to account for fluctuations caused by transmission over a mobile channel.