Collaborative Research: NeTS: Small: A Privacy-Aware Human-Centered QoE Assessment Framework for Immersive Videos

身临其境的视频，也称为360度视频，为观众提供了他们所处环境的完整视觉视角。到2025年，身临其境视频的全球市场预计将达到220亿美元。随着360度视频的日益流行，网络运营商和服务提供商越来越热衷于深入了解用户的体验质量(QOE)感知。然而，传统的针对二维视频设计的QOE模型无法捕捉到用户在观看这类新视频时独特的主观感受。为了弥合这一差距，该项目专注于构建以人为中心的模型，通过各种虚拟现实(VR)机载传感器利用“人的因素”。利用多模式感官读数，研究人员将利用用户感知体验的直接表示。这种方法捕捉到了在现有QOE模型中通常使用的客观系统参数经常遗漏的主观感受。该项目的成功将为用户在参与360度视频的同时，提供对细微差别的感知体验的深入了解。它将为利益相关者创造机会，实施针对用户的网络资源优化和视频流战略，最终提高个人客户满意度。研究成果将为以人为中心的传感和网络研究社区做出重大贡献，并使360度视频以外的众多VR应用受益。该项目涉及三个密切相关的研究推动力。第一个重点是研究如何从多模式感官读数中提取显著特征，以进行有效的QOE评估。为了对抗标签分布不均匀的数据集所产生的偏见，将开发新的方法来通过探索跨传感模式的有用信息来增强未被充分代表的数据。第二个重点是优化QOE评估的系统资源利用率。多个个性化模型将共享边缘服务器上的公共神经网络层，以实现资源高效的模型托管。在感官数据采集过程中，VR终端将采用自适应采样，以保持数据的有效性。第三个重点是在不牺牲QOE评估准确性的情况下设计数据隐私保护机制。在差异隐私框架下，该方法对多通道感知数据的数据相关性进行了形式化量化，这在很大程度上被以往的工作所忽视。这项拟议的研究是跨学科的，涵盖数据驱动的建模、传感和隐私保护计算。建议的机制和设计将通过测量活动、模拟和实验研究的组合进行彻底评估。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Immersive videos, also known as 360-degree videos, provide viewers with a complete visual perspective of their environment. The global market for immersive video is projected to reach 22 billion U.S. dollars by 2025. With the rising popularity of 360-degree videos, network operators and service providers are increasingly keen on gaining insights into users’ Quality of Experience (QoE) perception. However, the conventional QoE models designed for two dimensional videos fall short of capturing users’ unique subjective feelings in viewing this new type of video. To bridge this gap, the project focuses on constructing human-centered models that utilize “human factors” via a variety of virtual reality (VR) onboard sensors. Leveraging multi-modal sensory readings, the researchers will tap into a direct representation of users’ perceptual experiences. This approach captures subjective feelings often missed by the objective system parameters commonly used in existing QoE models. The success of this project will provide an in-depth understanding of users’ nuanced perceptual experience while engaging with 360-degree videos. It will create opportunities for stakeholders to implement user-specific network resource optimization and video streaming strategies, ultimately enhancing customer satisfaction on an individual basis. The research outcomes will significantly contribute to the human-centered sensing and networking research communities and benefit numerous VR applications beyond 360-degree videos.This project involves three closely related research thrusts. The first thrust investigates how to extract salient features from multi-modal sensory readings for effective QoE assessment. To combat bias arising from datasets with unevenly distributed labels, novel approaches will be developed to augment underrepresented data by exploring useful information across sensing modalities. The second thrust seeks to optimize system resource utilization for QoE assessment. Multiple personalized models will share common neural network layers at edge servers for resource-efficient model hosting. Adaptive sampling will be applied at VR terminals during sensory data acquisition to maintain data utility. The third thrust focuses on devising data privacy protection mechanisms without sacrificing QoE assessment accuracy. Under the differential privacy framework, the approach features formal quantification of data correlation of multi-modality sensory data, which has been largely overlooked by prior work. The proposed research is inter-disciplinary, which spans data-driven modeling, sensing, and privacy-preserving computing. The proposed mechanisms and designs will be thoroughly evaluated via a combination of measurement campaigns, simulations, and experimental studies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.