权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CNS Core: Medium: Collaborative Research: Scalable Dissemination and Navigation of Video 360 Content for Personalized Viewing

CNS 核心：媒介：协作研究：视频 360 内容的可扩展传播和导航以实现个性化观看

基本信息

批准号：
1900875
负责人：
Klara Nahrstedt
金额：
$ 40万
依托单位：
University of Illinois at Urbana-Champaign
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-10-01 至 2024-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1900875&HistoricalAwards=false
关键词：
CNS Core Medium Collaborative Research

项目摘要

360 video is a form of virtual reality (VR) that allows the viewer to experience media content in an immersive fashion. In contrast to traditional video, 360 video is recorded with a special camera that captures the complete surroundings from almost all directions. Viewers consuming such a video can select the direction they are looking at by using a pointing device on a regular display or through head movement using a head-mounted device. This new format allows a viewer to change their viewing direction when watching the video, e.g., a viewer can watch a sporting event from multiple perspectives on the field. However, creating, storing, and disseminating 360 videos at a large scale over the Internet poses significant challenges. These challenges are the focus of this project, which will develop a new system and framework, called mi360World, to enable smooth delivery of and interaction with 360 video by any user on the Internet. This project, if successful, will significantly improve 360 video delivery, and will enable new and much richer educational, training and entertainment experiences. It will also help train a new class of multimedia systems researchers and practitioners. The mechanisms for delivering a high-quality, personalized 360 video over the Internet to a globally distributed set of users is an unsolved scientific problem that entails the following challenges: 1) Ultra-high Bandwidth; 2) Ultra-low Delay; 3) View Adaptation (to user head movement); 4) Complex video metadata and delivery; 5) Video Quality of Experience (QoE). Traditional video QoE has seen extensive research over the years; however, what contributes to 360 video QoE is much less understood and will require conceiving of and measuring new metrics. The proposed mi360World system incorporates three major research thrusts to address the above challenges: A video creation thrust that enables personalized viewing by generating navigation graphs and cinematographic rules, while maintaining a high QoE and reducing cybersickness. The construction of navigation graphs and inclusion of cinematographic rules represent the main innovations of this project, and are encapsulated in a three-layered metadata representation of the 360 video: a transport layer, a semantic layer, and an interactive story-telling layer. The second thrust focuses on scalable distribution of 360 videos to a global set of diverse viewers, utilizing navigation graphs and cinematographic rules for highly efficient transition-predictive prefetching and caching. The third thrust focuses on QoE and has the goal of devising novel QoE metrics and evaluation methods to assess cybersickness. System architectures and algorithms will be extensively evaluated through simulation, emulation, and benchmarking using testbeds to assess the success of the proposed research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

360度视频是一种虚拟现实（VR）形式，允许观众以沉浸式方式体验媒体内容。与传统视频不同的是，360度视频是用一个特殊的摄像头录制的，几乎可以从各个方向捕捉到完整的周围环境。消费这种视频的观看者可以通过使用常规显示器上的指向设备或通过使用头戴式设备的头部移动来选择他们正在观看的方向。这种新格式允许观看者在观看视频时改变他们的观看方向，例如，观众可以从场地上的多个视角观看体育赛事。然而，在互联网上大规模地创建、存储和传播360度视频带来了巨大的挑战。这些挑战是该项目的重点，该项目将开发一个名为mi 360 World的新系统和框架，以使互联网上的任何用户都能顺利地交付360度视频并与之交互。该项目如果成功，将显著改善360度视频传输，并将带来更丰富的教育、培训和娱乐体验。它还将帮助培训一批新的多媒体系统研究人员和从业人员。通过互联网向全球分布的用户提供高质量、个性化360度视频的机制是一个尚未解决的科学问题，它带来了以下挑战：1）超高带宽; 2）超低延迟; 3）视图自适应（针对用户头部移动）; 4）复杂的视频元数据和交付; 5）视频体验质量（QOE）。多年来，传统视频QoE已经得到了广泛的研究;然而，对360视频QoE的贡献要少得多，需要构思和测量新的指标。拟议中的mi 360 World系统包含三个主要研究方向来解决上述挑战：视频创建方向，通过生成导航图和电影摄影规则实现个性化观看，同时保持高QoE并减少晕机。导航图的构建和电影摄影规则的纳入代表了该项目的主要创新，并封装在360度视频的三层元数据表示中：传输层，语义层和交互式讲故事层。第二个重点是将360个视频可扩展地分发给全球不同的观众，利用导航图和电影摄影规则进行高效的过渡预测预取和缓存。第三个重点是QoE，目标是设计新的QoE指标和评估方法来评估晕网。系统架构和算法将通过模拟、仿真和基准测试进行广泛的评估，以评估拟议研究的成功与否。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（5）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

360BroadView: Viewer Management for Viewport Prediction in 360-Degree Video Live Broadcast

DOI：
10.1145/3551626.3564939
发表时间：
2022-12
期刊：
Proceedings of the 4th ACM International Conference on Multimedia in Asia
影响因子：
0
作者：
Qian Zhou;Zhe Yang;Hongpeng Guo;Beitong Tian;K. Nahrstedt
通讯作者：
Qian Zhou;Zhe Yang;Hongpeng Guo;Beitong Tian;K. Nahrstedt

Navigation Graph for Tiled Media Streaming

DOI：
10.1145/3343031.3351021
发表时间：
2019-10
期刊：
Proceedings of the 27th ACM International Conference on Multimedia
影响因子：
0
作者：
Jounsup Park;K. Nahrstedt
通讯作者：
Jounsup Park;K. Nahrstedt

Ultra-Sparse 360-Degree Camera View Synthesis for Immersive Virtual Tourism

用于沉浸式虚拟旅游的超稀疏 360 度摄像机视图合成

DOI：
10.1109/mipr54900.2022.00057
发表时间：
2022
期刊：
2022 IEEE 5th International Conference on Multimedia Information Processing and Retrieval (MIPR
影响因子：
0
作者：
Zhou, Qian;Nahrstedt, Klara
通讯作者：
Nahrstedt, Klara

L3BOU: Low Latency, Low Bandwidth, Optimized Super-Resolution Backhaul for 360-Degree Video Streaming

L3BOU：低延迟、低带宽、优化的超分辨率回程，适用于 360 度视频流