Low-delay reliable video delivery for real-time immersive applications

近年来，视频设备、应用和系统的快速部署，这种增长只会加剧。大量的视频数据将在非常创新的应用中被流传输，例如现场沉浸式视频（例如，360-度视频），视觉远程机械控制，远程手术和实时车间通信。各种各样的交互式视频应用部署在异构网络上，确保视频数据的可靠传输，特别是在低延迟约束下，是极具挑战性的。具体地，无线网络上的传输错误和分组丢失导致显著的延迟和视觉质量下降。因此，在寻求为最终用户提供最佳体验质量的过程中，以快速可靠的方式传输视频内容的能力构成了现代网络（如5G）最关键的挑战之一。该研究计划的主要目的是提出新的解决方案，以确保在非常低的延迟限制下，在沉浸式视频应用的背景下，通过异构网络可靠地传输视频数据的问题。具体而言，申请人及其团队将创建有前途的方法来保护视频数据免受丢失和传输错误的影响，以及纠正和修复视频内容的方法。这将消除对重传的需要，重传对于非常低延迟的应用是不期望的或不适合的。在这一过程中，他们将进行五项研究。首先，他们将创建新的前向纠错（FEC）方法，适用于低延迟交互式视频应用。其次，他们将提出一种全新的方法，将他们新开发的用于纠正错误视频数据包的算法与这些新的FEC类相结合，使用循环冗余校验信息，以保护系统免受比特错误和数据包丢失的影响。第三，他们将开发新的机器学习方法，使FEC参数动态适应视频内容和不断变化的网络条件，以确保在满足延迟限制的同时实现无错误的视频重建。第四，他们将利用新的通用视频编码（VVC）/H.266标准中的编码工具，将这些技术应用于基于瓦片的360度视频的编码和交付。最后，他们将提出新的深度学习方法来评估受损和修复视频的质量。这项研究提供的进步将显著提高交互式和沉浸式视频服务以及其他延迟敏感型应用的质量和可用性，并使前所未有的媒体体验得以出现。至于他们以前的研究，加拿大将从其中技术的知识产权和商业化以及在快速增长的沉浸式视频领域培训高素质人员中受益匪浅。

Recent years have witnessed a rapid deployment of video devices, applications and systems, and this growth will only intensify. Large amounts of video data will be streamed in very innovative applications such as live immersive video (e.g., 360-degree video), visual remote machinery control, remote surgery and live inter-vehicular communications. A wide diversity of interactive video applications are deployed on heterogeneous networks, and ensuring the reliable delivery of video data, especially under low delay constraints, is highly challenging. Specifically, transmission errors and packet losses over wireless networks cause significant delays and visual quality degradation. Thus, in seeking to provide the best quality of experience to end users, the ability to transport video content in a fast and reliable manner constitutes one of the most crucial challenges for modern networks, such as 5G. This research program aims mainly to propose novel solutions to the problem of ensuring the reliable delivery of video data over heterogeneous networks, under very low delay constraints, in the context of immersive video applications. Specifically, the applicant and his team will create promising methods to protect video data against losses and transmission errors, in addition to methods to correct and repair video content. These will remove the need for retransmissions, which are undesirable or unsuitable for very low-delay applications. In this quest, they will pursue five research tracks. First, they will create new classes of forward error correction (FEC) methods suitable for low-delay interactive video applications. Second, they will propose a radically new approach combining their newly developed algorithms for correcting erroneous video packets, using the Cyclic Redundancy Check information, with these new FEC classes, to protect the system against both bit errors and packets losses. Third, they will develop new machine learning approaches to dynamically adapt the FEC parameters to the video content and ever-changing network conditions in order to ensure error-free video reconstruction while meeting delay constraints. Fourth, they will apply these technologies to the coding and delivery of tiles-based 360-degree video by exploiting coding tools found in the new versatile video coding (VVC)/H.266 standard. Finally, they will propose new deep-learning approaches to evaluate the quality of damaged and repaired videos. The advances provided by this research will significantly improve the quality and usability of interactive and immersive video services, as well as other delay-sensitive applications, and enable the emergence of unprecedented media experiences. As for their previous research, Canada will benefit strongly from the intellectual property and commercialization of the technologies therein and from the training of highly qualified personnel in the fast-growing field of immersive video.