Adaptive Delivery and Analysis of Mixed Reality Content

点击翻译按钮获取中文摘要

Recent advances in computing, sensing, and display technologies have enabled new types of multimedia content such as 360-degree videos, virtual reality (VR) and augmented reality (AR), and hyperspectral videos; we collectively refer to them as Mixed Reality (MR) content. MR content and systems are expected to change many aspects of our everyday life, from entertainment and training services to industrial, medical and military applications. Realizing their full potential, however, requires addressing several research challenges. For example, MR content requires transferring substantial amounts of data, and must support users' interactivity and navigation of various parts of the content. Given the continuing growth of the Internet traffic and that most of it carries multimedia content, a vast amount of multimedia traffic will need to be efficiently delivered to numerous, diverse, stationary and mobile users. We address this challenge by designing scalable and adaptive streaming algorithms that: (i) support user interactivities with minimal bandwidth overheads, (ii) leverage the characteristics of modern backbone networks (e.g., flexibility of managing the network resources using software-defined networking (SDN) concepts), and (iii) enable full utilization of next-generation cellular networks features (e.g., virtualized base stations, heterogeneous cells, and multicast support). We will design algorithms to optimize the quality of experience for users, reduce the load on the network, and crucially, minimize the energy consumption of battery-powered mobile devices. In addition, for MR content to be useful especially in military and industrial applications, a better understanding of such feature-rich content is essential. We will design methods to analyze and magnify hidden information in MR content, especially hyperspectral videos. Hyperspectral videos provide substantially more information than traditional videos, because they capture a scene across many wavelength bands, not only in the visible light range as current videos. They can, for example, measure the temperature of a remote object and identify its material composition. We will design methods to magnify the signals in different wavelength bands in hyperspectral videos, and to prioritize the transmission of various components of such complex videos based on the application needs and available bandwidth. ***This research program will design algorithms and systems to enable next-generation immersive multimedia services, which will potentially have substantial economic and social benefits for Canadians. It will train several students in the theoretical and practical aspects of AR/VR, SDN-enabled networks, 5G cellular networks, and analysis and magnification of hyperspectral videos, which are all cutting-edge and active research topics of high demand by Canadian high-tech companies.