CAREER: System Research to Enable Practical Immersive Streaming: From 360-Degree Towards Volumetric Video Delivery

职业:实现实用沉浸式流媒体的系统研究:从 360 度到立体视频传输

基本信息

  • 批准号:
    2200048
  • 负责人:
  • 金额:
    $ 48.62万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
    Continuing Grant
  • 财政年份:
    2021
  • 资助国家:
    美国
  • 起止时间:
    2021-10-01 至 2025-03-31
  • 项目状态:
    未结题

项目摘要

Immersive video technologies allow users to freely explore remote or virtual environments. For example, with 360-degree videos, users can view the scene from any orientation; with volumetric videos, users can control not only the view orientation but also the camera position. Such highly immersive content has applications in entertainment, medicine, education, manufacturing, and e-commerce, to name just a few areas. However, current video streaming infrastructure cannot fully support these emerging formats’ high bandwidth, low latency, and high storage requirements. This project aims to address challenges in efficient transmission and storage of immersive video streams by proposing new, efficient representations of this content. These representations can both be adapted to fit users’ viewing behaviors and can be generated at the low-latencies required for real-time streaming applications. If successful, the proposed research will enable higher quality immersive streaming than is possible with current systems, further enabling useful immersive streaming experiences. This project investigates techniques for improving efficiency of two specific immersive streaming applications. For real-time 360-degree video, the project will build a system to generate area-of-focus projections in real-time. These area-of-focus projections are selected to align the high-quality focus area with a predicted user view. Low-latency generation is achieved using graphics processing units at nearby edge or cloud servers. For volumetric video streaming, this project aims to create both a storage- and bandwidth-efficient representation of the video. The representation consists of both area-of-focus versions of the video at a selected set of points within the volume as well as patches to cover dis-occluded pixels, allowing high-quality video to be delivered to users positioned anywhere in the scene. The proposed system further uses a Hypertext Transfer Protocol Version 2 (HTTP/2) transmission approach to support bandwidth-efficient delivery of this representation. To more precisely measure the user’s true experience of immersive streams, this project will also create immersive video streaming datasets and investigate a new quality metric. This new metric will use novel approaches to correlate user-perceived qualities with visual artifacts introduced due to encoding and network transmission. Artifacts produced as a result of this project, including publications, code, and datasets, will be made publicly available at http://www.cs.binghamton.edu/~yaoliu/ImmersiveStreaming/. These artifacts will be maintained for at least five years after completion of the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
沉浸式视频技术允许用户自由探索远程或虚拟环境。例如,使用360度视频,用户可以从任何方向查看场景;使用体积视频,用户不仅可以控制查看方向,还可以控制摄像头位置。这种高度身临其境的内容在娱乐、医疗、教育、制造和电子商务等领域都有应用。然而,目前的视频流基础设施不能完全支持这些新兴格式的高带宽、低延迟和高存储要求。该项目旨在通过提出沉浸式视频流的新的、高效的表示来解决在高效传输和存储沉浸式视频流方面的挑战。这些表示既可以适应用户的观看行为,也可以在实时流应用所需的低延迟下生成。如果成功,拟议的研究将实现比当前系统更高质量的沉浸式流媒体,进一步实现有用的沉浸式流媒体体验。本项目研究提高两个特定沉浸式流媒体应用程序的效率的技术。对于实时360度视频,该项目将建立一个系统来实时生成焦点区域投影。选择这些焦点区域投影以使高质量焦点区域与预测的用户视图对准。使用附近边缘或云服务器上的图形处理单元可实现低延迟生成。对于体积视频流,该项目的目标是创建存储和带宽高效的视频表示。该表示包括体积内选定点集的视频的焦点区域版本以及覆盖非遮挡像素的补丁,从而允许将高质量视频传送给场景中任何位置的用户。所提出的系统还使用超文本传输协议版本2(HTTP/2)传输方法来支持该表示的带宽高效传递。为了更准确地衡量用户对沉浸式视频流的真实体验,该项目还将创建沉浸式视频流数据集,并调查新的质量指标。这一新的衡量标准将使用新的方法将用户感知的质量与由于编码和网络传输而引入的视觉伪像关联起来。由于该项目而产生的构件,包括出版物、代码和数据集,将在http://www.cs.binghamton.edu/~yaoliu/ImmersiveStreaming/.上公开提供这些文物将在项目完成后至少保留五年。这一奖项反映了NSF的法定使命,并通过使用基金会的智力优势和更广泛的影响审查标准进行评估,被认为值得支持。

项目成果

期刊论文数量(5)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Exploring Spherical Autoencoder for Spherical Video Content Processing
EVASR: Edge-Based Video Delivery with Salience-Aware Super-Resolution
EVASR:具有显着性感知超分辨率的基于边缘的视频传输
VertexShuffle-Based Spherical Super-Resolution for 360-Degree Videos
Security-Preserving Live 3D Video Surveillance
VQBA: Visual-Quality-Driven Bit Allocation for Low-Latency Point Cloud Streaming
VQBA:低延迟点云流的视觉质量驱动的比特分配
  • DOI:
    10.1145/3581783.3612486
  • 发表时间:
    2023
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Wang, Shuoqian;Zhu, Mufeng;Li, Na;Xiao, Mengbai;Liu, Yao
  • 通讯作者:
    Liu, Yao
{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Yao Liu其他文献

NaTiSi_2O_6/C Composite as a Novel Anode Material for Lithium-Ion Batteries
NaTiSi_2O_6/C复合材料作为锂离子电池新型负极材料
  • DOI:
    10.3866/pku.whxb201912030
  • 发表时间:
    2020
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Kun Liu;Yao Liu;Haifeng Zhu;Xiaoli Dong;Yonggang Wang;Congxiao Wang;Yongyao Xia
  • 通讯作者:
    Yongyao Xia
1. Reconstruction of Hematopoietic Inductive Microenvironment after Transplantation of VCAM-1-Modified Human Umbilical Cord Blood Stromal Cells
1. VCAM-1修饰的人脐带血基质细胞移植后造血诱导微环境的重建
  • DOI:
  • 发表时间:
    2012
  • 期刊:
  • 影响因子:
    3.7
  • 作者:
    Yao Liu;Xing-hua Chen;Ying-jian Si;Zhong-jun Li;Lei Gao;Li Gao;Cheng Zhang;Xi Zhang(通讯作者)
  • 通讯作者:
    Xi Zhang(通讯作者)
Polymer Design to Promote Low Work Function Surfaces in Organic Electronics
聚合物设计促进有机电子中的低功函数表面
  • DOI:
    10.1016/j.progpolymsci.2020.101222
  • 发表时间:
    2020-02
  • 期刊:
  • 影响因子:
    27.1
  • 作者:
    Yao Wu;Yao Liu;Todd Emrick;Thomas P.Russell
  • 通讯作者:
    Thomas P.Russell
The properties of the inner disk around HL Tau: Multi-wavelength modeling of the dust emission
HL Tau 周围内盘的特性:粉尘排放的多波长建模
  • DOI:
    10.1051/0004-6361/201629786
  • 发表时间:
    2017-08
  • 期刊:
  • 影响因子:
    6.5
  • 作者:
    Yao Liu;Thomas Henning;Carlos Carrasco-González;Claire J. Ch;ler;Hendrik Linz;Til Birnstiel;Roy van Boekel;Laura M. Pérez;Mario Flock;Leonardo Testi;Luis F. Rodríguez;Roberto Galván-Madrid
  • 通讯作者:
    Roberto Galván-Madrid
Landsat-Satellite-Based Analysis of Long-Term Temporal Spatial Dynamics of Cyanobacterial Blooms: A Case Study in Taihu Lake
基于陆地卫星的蓝藻水华长期时空动态分析:以太湖为例
  • DOI:
    10.3390/land11122197
  • 发表时间:
    2022-12
  • 期刊:
  • 影响因子:
    3.9
  • 作者:
    Jingtai Li;Yao Liu;Siying Xie;Min Li;Li Chen;Cuiling Wu;D;an Yan;Zhaoqing Luan
  • 通讯作者:
    Zhaoqing Luan

Yao Liu的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Yao Liu', 18)}}的其他基金

Collaborative Research: CNS Core: Small: From Capture to Consumption: System Challenges in Pervasive 360-Degree Video Sharing
合作研究:CNS 核心:小型:从捕获到消费:普遍 360 度视频共享的系统挑战
  • 批准号:
    2200042
  • 财政年份:
    2021
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Standard Grant
CAREER: System Research to Enable Practical Immersive Streaming: From 360-Degree Towards Volumetric Video Delivery
职业:实现实用沉浸式流媒体的系统研究:从 360 度到立体视频传输
  • 批准号:
    1943250
  • 财政年份:
    2020
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Continuing Grant
SaTC: TTP: Small: Creating Content Verification Tools to Protect Document Integrity
SaTC:TTP:小型:创建内容验证工具来保护文档完整性
  • 批准号:
    2024300
  • 财政年份:
    2020
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Standard Grant
Collaborative Research: CNS Core: Small: From Capture to Consumption: System Challenges in Pervasive 360-Degree Video Sharing
合作研究:CNS 核心:小型:从捕获到消费:普遍 360 度视频共享的系统挑战
  • 批准号:
    2007176
  • 财政年份:
    2020
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Standard Grant
Student Travel Support for IEEE CNS 2019
IEEE CNS 2019 学生旅行支持
  • 批准号:
    1917320
  • 财政年份:
    2019
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Standard Grant
Student Travel and Student PC Meeting Support for IEEE S&P 2019
IEEE S 的学生旅行和学生 PC 会议支持
  • 批准号:
    1852720
  • 财政年份:
    2019
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Standard Grant
CSR: Small: A Unified Approach Toward User-specific Improvements of Quality of Experience for Video Streaming
CSR:小:针对特定用户改进视频流体验质量的统一方法
  • 批准号:
    1618931
  • 财政年份:
    2016
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Standard Grant
CAREER: A Pathway towards Channel Camouflage and Manipulation Techniques for Wireless Security
职业:无线安全信道伪装和操纵技术的途径
  • 批准号:
    1553304
  • 财政年份:
    2016
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Continuing Grant
TWC: Small: Techniques and Tools for Enforcing Proximity-based Policies in Wireless Systems
TWC:小型:在无线系统中实施基于邻近策略的技术和工具
  • 批准号:
    1527144
  • 财政年份:
    2015
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Standard Grant

相似国自然基金

Data-driven Recommendation System Construction of an Online Medical Platform Based on the Fusion of Information
  • 批准号:
  • 批准年份:
    2024
  • 资助金额:
    万元
  • 项目类别:
    外国青年学者研究基金项目
TBX1/LKB1轴阻断system Xc活性调控AML细胞铁死亡的机制研究
  • 批准号:
  • 批准年份:
    2024
  • 资助金额:
    15.0 万元
  • 项目类别:
    省市级项目
TET2通过调控BAP1-System Xc-轴促进紫拉非尼诱导的肝细胞癌铁死亡的机制研究
  • 批准号:
  • 批准年份:
    2024
  • 资助金额:
    0 万元
  • 项目类别:
    青年科学基金项目
P3H1通过ATF4/System Xc-轴抑制肾癌铁死亡和抗肿瘤免疫反应的作用及机制研究
  • 批准号:
    82372704
  • 批准年份:
    2023
  • 资助金额:
    49 万元
  • 项目类别:
    面上项目
基于PNO1介导system Xc-/GSH途径调控肠上皮细胞自噬依赖性铁死亡探讨加味胶七散治疗溃疡性结肠炎的机制
  • 批准号:
    82304982
  • 批准年份:
    2023
  • 资助金额:
    30 万元
  • 项目类别:
    青年科学基金项目
基于单细胞测序探讨淫羊藿苷对Erastin诱导髓核细胞铁死亡相关system-Xc/GSH/GPX4分子轴线的调控作用
  • 批准号:
    82360947
  • 批准年份:
    2023
  • 资助金额:
    33 万元
  • 项目类别:
    地区科学基金项目
miR-198 靶向 Nrf2 抑制 System Xc-通路调控滋养细胞铁死亡在子痫前期中的机制
  • 批准号:
    2022JJ70123
  • 批准年份:
    2022
  • 资助金额:
    0.0 万元
  • 项目类别:
    省市级项目
BAP1介导H2B去泛素化抑制System Xc-在蛛网膜下腔出血神经元铁死亡中的作用和机制研究
  • 批准号:
  • 批准年份:
    2021
  • 资助金额:
    55 万元
  • 项目类别:
    面上项目

相似海外基金

Collaborative Research: The AGEP Massachusetts State University System Equity-Minded Model for Recruiting and Advancing Early Career Faculty in the STEM Professoriate
合作研究:AGEP 马萨诸塞州立大学系统公平思维模型,用于招募和提升 STEM 教授中的早期职业教师
  • 批准号:
    2149883
  • 财政年份:
    2022
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Continuing Grant
Collaborative Research: The AGEP Massachusetts State University System Equity-Minded Model for Recruiting and Advancing Early Career Faculty in the STEM Professoriate
合作研究:AGEP 马萨诸塞州立大学系统公平思维模型,用于招募和提升 STEM 教授中的早期职业教师
  • 批准号:
    2149858
  • 财政年份:
    2022
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Continuing Grant
Prize 202209PJT - Health Services and Policy Research Rising Star Early Career Award: Building system capacity for home visits for patients near the end of life: A mixed methods study
奖 202209PJT - 卫生服务和政策研究新星早期职业奖:建立临终患者家访的系统能力:混合方法研究
  • 批准号:
    477694
  • 财政年份:
    2022
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Operating Grants
Collaborative Research: The AGEP Massachusetts State University System Equity-Minded Model for Recruiting and Advancing Early Career Faculty in the STEM Professoriate
合作研究:AGEP 马萨诸塞州立大学系统公平思维模型,用于招募和提升 STEM 教授中的早期职业教师
  • 批准号:
    2149849
  • 财政年份:
    2022
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Continuing Grant
CAREER: Mechanistic dissection of sleep and behavioral arousal: A model system approach to interdisciplinarity in neuroscience research and education
职业:睡眠和行为唤醒的机制剖析:神经科学研究和教育跨学科的模型系统方法
  • 批准号:
    2042873
  • 财政年份:
    2021
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Continuing Grant
CAREER: System Research to Enable Practical Immersive Streaming: From 360-Degree Towards Volumetric Video Delivery
职业:实现实用沉浸式流媒体的系统研究:从 360 度到立体视频传输
  • 批准号:
    1943250
  • 财政年份:
    2020
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Continuing Grant
CAREER: The Role of Groundwater Storage in Earth System Dynamics; Research to Improve Understanding of Current Hydrologic Regimes and Future Climate Response
职业:地下水储存在地球系统动力学中的作用;
  • 批准号:
    1945195
  • 财政年份:
    2020
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Continuing Grant
Research on Career Path Training for Child Care Teachers: Development of a Training System Conscious of Individual Characteristics
幼儿教师职业路径培训研究:构建具有个体特色的培训体系
  • 批准号:
    18K02458
  • 财政年份:
    2018
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Grant-in-Aid for Scientific Research (C)
Research on construction of career development system for Certified Care Workers
护理福祉士职业发展体系构建研究
  • 批准号:
    22730435
  • 财政年份:
    2010
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Grant-in-Aid for Young Scientists (B)
Historical Research on the Relationship between Ichizou Hayashi's Career and the Houmen-iin System after his Resignation as Governor of Osaka
林一三卸任大阪府知事后的仕途与后免院制度的关系史学研究
  • 批准号:
    21530618
  • 财政年份:
    2009
  • 资助金额:
    $ 48.62万
  • 项目类别:
    Grant-in-Aid for Scientific Research (C)
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了