Adaptive Delivery and Analysis of Mixed Reality Content

混合现实内容的自适应交付和分析

• 批准号: RGPIN-2018-05048
• 负责人: Hefeeda, Mohamed
• 金额: $ 2.99万
• 依托单位: Simon Fraser University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=688185
• 关键词: Adaptive; Delivery; Analysis; Mixed; Reality

Recent advances in computing, sensing, and display technologies have enabled new types of multimedia content such as 360-degree videos, virtual reality (VR) and augmented reality (AR), and hyperspectral videos; we collectively refer to them as Mixed Reality (MR) content. MR content and systems are expected to change many aspects of our everyday life, from entertainment and training services to industrial, medical and military applications. Realizing their full potential, however, requires addressing several research challenges. For example, MR content requires transferring substantial amounts of data, and must support users' interactivity and navigation of various parts of the content. Given the continuing growth of the Internet traffic and that most of it carries multimedia content, a vast amount of multimedia traffic will need to be efficiently delivered to numerous, diverse, stationary and mobile users. We address this challenge by designing scalable and adaptive streaming algorithms that: (i) support user interactivities with minimal bandwidth overheads, (ii) leverage the characteristics of modern backbone networks (e.g., flexibility of managing the network resources using software-defined networking (SDN) concepts), and (iii) enable full utilization of next-generation cellular networks features (e.g., virtualized base stations, heterogeneous cells, and multicast support). We will design algorithms to optimize the quality of experience for users, reduce the load on the network, and crucially, minimize the energy consumption of battery-powered mobile devices. In addition, for MR content to be useful especially in military and industrial applications, a better understanding of such feature-rich content is essential. We will design methods to analyze and magnify hidden information in MR content, especially hyperspectral videos. Hyperspectral videos provide substantially more information than traditional videos, because they capture a scene across many wavelength bands, not only in the visible light range as current videos. They can, for example, measure the temperature of a remote object and identify its material composition. We will design methods to magnify the signals in different wavelength bands in hyperspectral videos, and to prioritize the transmission of various components of such complex videos based on the application needs and available bandwidth. ***This research program will design algorithms and systems to enable next-generation immersive multimedia services, which will potentially have substantial economic and social benefits for Canadians. It will train several students in the theoretical and practical aspects of AR/VR, SDN-enabled networks, 5G cellular networks, and analysis and magnification of hyperspectral videos, which are all cutting-edge and active research topics of high demand by Canadian high-tech companies.

计算、传感和显示技术的最新进展使新型多媒体内容成为可能，例如360度视频、虚拟现实(VR)和增强现实(AR)以及高光谱视频；我们统称它们为混合现实(MR)内容。MR的内容和系统有望改变我们日常生活的方方面面，从娱乐和培训服务到工业、医疗和军事应用。然而，实现它们的全部潜力需要解决几个研究挑战。例如，MR内容需要传输大量数据，并且必须支持用户对内容的各个部分进行交互和导航。鉴于互联网流量的持续增长以及其中大部分承载多媒体内容，大量的多媒体流量将需要高效地提供给众多的、多样化的、固定的和移动的用户。我们通过设计可扩展和自适应的流传输算法来解决这一挑战：(I)以最小的带宽开销支持用户交互，(Ii)利用现代骨干网络的特性(例如，使用软件定义的联网(SDN)概念管理网络资源的灵活性)，以及(Iii)能够充分利用下一代蜂窝网络特征(例如，虚拟基站、异类小区和多播支持)。我们将设计算法来优化用户体验质量，减少网络负载，最重要的是最大限度地减少电池供电的移动设备的能耗。此外，要使MR内容在军事和工业应用中特别有用，对这种功能丰富的内容的更好理解是至关重要的。我们将设计方法来分析和放大MR内容中的隐藏信息，特别是高光谱视频。高光谱视频提供了比传统视频多得多的信息，因为它们捕捉到了跨多个波长段的场景，而不仅仅是像当前视频那样在可见光范围内。例如，它们可以测量远程物体的温度并识别其材料组成。我们将设计方法放大高光谱视频中不同波长段的信号，并根据应用需求和可用带宽对此类复杂视频的各个分量的传输进行优先排序。*这项研究计划将设计算法和系统，以实现下一代沉浸式多媒体服务，这可能会为加拿大人带来巨大的经济和社会效益。它将在AR/VR、SDN网络、5G蜂窝网络、高光谱视频的分析和放大等理论和实践方面培训几名学生，这些都是加拿大高科技公司高需求的前沿和活跃的研究课题。