CIF: Medium: Mobile multiview video: compression, rendering, and transmission

CIF：中：移动多视图视频：压缩、渲染和传输

• 批准号: 1160832
• 负责人: Pamela Cosman
• 金额: $ 118.82万
• 依托单位: University of California-San Diego
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-04-01 至 2016-03-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1160832&HistoricalAwards=false
• 关键词: CIF; Medium; Mobile; multiview; video

The prominence of 3D video technology has skyrocketed. The 2009 movie Avatar in 3D became the highest grossing movie of all time. Such a movie requires left and right views of a scene. Many video games which provide a 3D experience require multiple views of a scene. Such data is costly to store and to transmit.This research studies how to efficiently compress multiple-view video data, how to allow the scene to be viewed from any angle at different levels of precision, and how to reliably transmit the data over mobile wireless channels. This research has important applications in science education, traffic monitoring, and surveillance and security.This research studies efficient encoding, rendering, and transmission of multi-view video, aiming for robust performance at arbitrary speeds of mobile units. The research is applicable both to videos of the real world taken with multiple cameras, and to rendered videos. The investigators study left/right view coding such as in the H.264 MVC standard, and view+depth coding. The latter approach is enhanced by encoding the error signal between the original view and its decoder-synthesized version. To optimally design the system, the techniques use cross-layer optimization, in which physical-layer channel-state information and application-layer distortion-rate or slice-priority information are exploited. Whenever multiple views are rendered from an underlying 3D virtual world, the application's bit requirements can be hugely altered by rendering parameters which affect the content and level of detail of the scene. The investigators study user-experience models to quantify the relationship between rendering parameters and user satisfaction, and develop a channel-aware adaptive encoding and rendering algorithm to account for fluctuations caused by transmission over a mobile channel.

3D 视频技术的重要性突飞猛进。 2009 年 3D 电影《阿凡达》成为有史以来票房最高的电影。这样的电影需要场景的左视图和右视图。许多提供 3D 体验的视频游戏需要场景的多个视图。这些数据的存储和传输成本很高。本研究研究如何有效地压缩多视图视频数据，如何允许从任何角度以不同的精度水平观看场景，以及如何通过移动无线信道可靠地传输数据。这项研究在科学教育、交通监控以及监视和安全方面具有重要的应用。这项研究研究了多视图视频的高效编码、渲染和传输，旨在在移动设备的任意速度下实现鲁棒的性能。该研究既适用于使用多个摄像机拍摄的现实世界视频，也适用于渲染视频。研究人员研究了 H.264 MVC 标准中的左/右视图编码以及视图+深度编码。通过对原始视图与其解码器合成版本之间的误差信号进行编码来增强后一种方法。为了优化系统设计，这些技术使用跨层优化，其中利用物理层信道状态信息和应用层失真率或切片优先级信息。 每当从底层 3D 虚拟世界渲染多个视图时，应用程序的位要求可能会因影响场景内容和细节级别的渲染参数而发生巨大变化。 研究人员研究用户体验模型，以量化渲染参数和用户满意度之间的关系，并开发一种通道感知的自适应编码和渲染算法，以解决移动通道传输引起的波动。