Collaborative Research: NeTS: Small: A Privacy-Aware Human-Centered QoE Assessment Framework for Immersive Videos

沉浸式视频，也被称为360度视频，为观众提供了一个完整的环境视觉视角。据预测，到2025年，全球沉浸式视频市场规模将达到220亿美元。随着360度视频的日益普及，网络运营商和服务提供商越来越热衷于深入了解用户的体验质量（QoE）感知。然而，传统的为二维视频设计的QoE模型无法捕捉到用户在观看这种新型视频时独特的主观感受。为了弥补这一差距，该项目专注于构建以人为中心的模型，通过各种虚拟现实（VR）车载传感器利用“人为因素”。利用多模态感官读数，研究人员将利用用户感知体验的直接表现。这种方法捕获了在现有QoE模型中通常使用的客观系统参数所忽略的主观感受。该项目的成功将在360度视频中深入了解用户细微的感知体验。它将为利益相关者创造机会，实施针对用户的网络资源优化和视频流策略，最终提高个人客户满意度。该研究成果将为以人为中心的传感和网络研究社区做出重大贡献，并使360度视频以外的许多VR应用受益。这个项目涉及三个密切相关的研究重点。第一个重点研究如何从多模态感官读数中提取显著特征，以进行有效的QoE评估。为了对抗标签分布不均的数据集产生的偏见，将开发新的方法，通过探索跨传感模式的有用信息来增加代表性不足的数据。第二个推动力寻求优化QoE评估的系统资源利用。多个个性化模型将在边缘服务器上共享公共神经网络层，以实现资源高效的模型托管。自适应采样将在VR终端的感官数据采集过程中应用，以保持数据的实用性。第三个重点是在不牺牲QoE评估准确性的情况下设计数据隐私保护机制。在差分隐私框架下，该方法对多模态感官数据的数据相关性进行了形式化量化，这在很大程度上被以往的工作所忽视。提出的研究是跨学科的，涵盖数据驱动建模，传感和隐私保护计算。提出的机制和设计将通过测量活动，模拟和实验研究的组合进行彻底评估。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Immersive videos, also known as 360-degree videos, provide viewers with a complete visual perspective of their environment. The global market for immersive video is projected to reach 22 billion U.S. dollars by 2025. With the rising popularity of 360-degree videos, network operators and service providers are increasingly keen on gaining insights into users’ Quality of Experience (QoE) perception. However, the conventional QoE models designed for two dimensional videos fall short of capturing users’ unique subjective feelings in viewing this new type of video. To bridge this gap, the project focuses on constructing human-centered models that utilize “human factors” via a variety of virtual reality (VR) onboard sensors. Leveraging multi-modal sensory readings, the researchers will tap into a direct representation of users’ perceptual experiences. This approach captures subjective feelings often missed by the objective system parameters commonly used in existing QoE models. The success of this project will provide an in-depth understanding of users’ nuanced perceptual experience while engaging with 360-degree videos. It will create opportunities for stakeholders to implement user-specific network resource optimization and video streaming strategies, ultimately enhancing customer satisfaction on an individual basis. The research outcomes will significantly contribute to the human-centered sensing and networking research communities and benefit numerous VR applications beyond 360-degree videos.This project involves three closely related research thrusts. The first thrust investigates how to extract salient features from multi-modal sensory readings for effective QoE assessment. To combat bias arising from datasets with unevenly distributed labels, novel approaches will be developed to augment underrepresented data by exploring useful information across sensing modalities. The second thrust seeks to optimize system resource utilization for QoE assessment. Multiple personalized models will share common neural network layers at edge servers for resource-efficient model hosting. Adaptive sampling will be applied at VR terminals during sensory data acquisition to maintain data utility. The third thrust focuses on devising data privacy protection mechanisms without sacrificing QoE assessment accuracy. Under the differential privacy framework, the approach features formal quantification of data correlation of multi-modality sensory data, which has been largely overlooked by prior work. The proposed research is inter-disciplinary, which spans data-driven modeling, sensing, and privacy-preserving computing. The proposed mechanisms and designs will be thoroughly evaluated via a combination of measurement campaigns, simulations, and experimental studies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.