CIF: Medium: Mobile multiview video: compression, rendering, and transmission

3D视频技术的显赫地位直线上升。2009年的3D电影《阿凡达》成为有史以来票房最高的电影。这样的电影需要场景的左视图和右视图。许多提供3D体验的视频游戏需要一个场景的多个视图。这类数据的存储和传输成本很高。本研究研究如何有效地压缩多视角视频数据，如何让场景以不同的精度从任何角度观看，以及如何在移动无线信道上可靠地传输数据。该研究在科学教育、交通监控、监控与安全等领域有着重要的应用。本研究旨在研究多视点视频的高效编码、渲染和传输，目标是在任意移动单元的速度下获得稳健的性能。这项研究既适用于用多个摄像头拍摄的真实世界的视频，也适用于渲染的视频。研究人员研究了诸如H.264 MVC标准中的左/右视图编码以及视点+深度编码。后一种方法通过对原始视图和其解码器合成版本之间的误差信号进行编码来增强。为了对系统进行优化设计，该技术使用跨层优化，其中利用了物理层的信道状态信息和应用层的失真率或片优先级信息。无论何时从底层的3D虚拟世界渲染多个视图，都可以通过渲染影响场景内容和细节级别的参数来极大地改变应用程序的位要求。研究人员研究了用户体验模型，以量化渲染参数和用户满意度之间的关系，并开发了一种通道感知的自适应编码和渲染算法，以解决移动通道上传输引起的波动。

The prominence of 3D video technology has skyrocketed. The 2009 movie Avatar in 3D became the highest grossing movie of all time. Such a movie requires left and right views of a scene. Many video games which provide a 3D experience require multiple views of a scene. Such data is costly to store and to transmit.This research studies how to efficiently compress multiple-view video data, how to allow the scene to be viewed from any angle at different levels of precision, and how to reliably transmit the data over mobile wireless channels. This research has important applications in science education, traffic monitoring, and surveillance and security.This research studies efficient encoding, rendering, and transmission of multi-view video, aiming for robust performance at arbitrary speeds of mobile units. The research is applicable both to videos of the real world taken with multiple cameras, and to rendered videos. The investigators study left/right view coding such as in the H.264 MVC standard, and view+depth coding. The latter approach is enhanced by encoding the error signal between the original view and its decoder-synthesized version. To optimally design the system, the techniques use cross-layer optimization, in which physical-layer channel-state information and application-layer distortion-rate or slice-priority information are exploited. Whenever multiple views are rendered from an underlying 3D virtual world, the application's bit requirements can be hugely altered by rendering parameters which affect the content and level of detail of the scene. The investigators study user-experience models to quantify the relationship between rendering parameters and user satisfaction, and develop a channel-aware adaptive encoding and rendering algorithm to account for fluctuations caused by transmission over a mobile channel.