Low-delay reliable video delivery for real-time immersive applications

近年来，视频设备、应用程序和系统的部署迅速，而且这种增长只会加剧。大量视频数据将在非常创新的应用中流传输，例如实时沉浸式视频(例如360度视频)、视觉远程机械控制、远程手术和实时车载通信。各种各样的交互式视频应用部署在不同的网络上，确保视频数据的可靠传输，特别是在低延迟限制下，是非常具有挑战性的。具体地说，无线网络上的传输错误和分组丢失会导致显著的延迟和视觉质量下降。因此，在寻求为最终用户提供最佳质量的体验时，以快速可靠的方式传输视频内容的能力构成了5G等现代网络面临的最关键挑战之一。该研究项目的主要目的是在沉浸式视频应用的背景下，提出新的解决方案，以确保在非常低的延迟约束下，在不同的网络上可靠地传输视频数据。具体地说，除了纠正和修复视频内容的方法外，申请者和他的团队将创造出有希望的方法来保护视频数据免受丢失和传输错误的影响。这将消除对重传的需要，重传是不希望的或不适合非常低延迟的应用。在这次探索中，他们将进行五项研究。首先，他们将创建适用于低延迟交互式视频应用的新类别的前向纠错(FEC)方法。其次，他们将提出一种全新的方法，将他们最新开发的使用循环冗余校验信息纠正错误视频分组的算法与这些新的FEC类别相结合，以保护系统免受误码和分组丢失。第三，他们将开发新的机器学习方法，以动态调整FEC参数以适应视频内容和不断变化的网络条件，以确保在满足延迟限制的同时无错误地重建视频。第四，他们将通过利用新的通用视频编码(VVC)/H.266标准中的编码工具，将这些技术应用于基于块的360度视频的编码和交付。最后，他们将提出新的深度学习方法来评估损坏和修复的视频的质量。这项研究提供的进步将显著提高交互式和沉浸式视频服务以及其他对延迟敏感的应用程序的质量和可用性，并使前所未有的媒体体验得以出现。至于他们以前的研究，加拿大将从其中技术的知识产权和商业化以及在快速增长的身临其境视频领域培训高素质人员中受益匪浅。

Recent years have witnessed a rapid deployment of video devices, applications and systems, and this growth will only intensify. Large amounts of video data will be streamed in very innovative applications such as live immersive video (e.g., 360-degree video), visual remote machinery control, remote surgery and live inter-vehicular communications. A wide diversity of interactive video applications are deployed on heterogeneous networks, and ensuring the reliable delivery of video data, especially under low delay constraints, is highly challenging. Specifically, transmission errors and packet losses over wireless networks cause significant delays and visual quality degradation. Thus, in seeking to provide the best quality of experience to end users, the ability to transport video content in a fast and reliable manner constitutes one of the most crucial challenges for modern networks, such as 5G. This research program aims mainly to propose novel solutions to the problem of ensuring the reliable delivery of video data over heterogeneous networks, under very low delay constraints, in the context of immersive video applications. Specifically, the applicant and his team will create promising methods to protect video data against losses and transmission errors, in addition to methods to correct and repair video content. These will remove the need for retransmissions, which are undesirable or unsuitable for very low-delay applications. In this quest, they will pursue five research tracks. First, they will create new classes of forward error correction (FEC) methods suitable for low-delay interactive video applications. Second, they will propose a radically new approach combining their newly developed algorithms for correcting erroneous video packets, using the Cyclic Redundancy Check information, with these new FEC classes, to protect the system against both bit errors and packets losses. Third, they will develop new machine learning approaches to dynamically adapt the FEC parameters to the video content and ever-changing network conditions in order to ensure error-free video reconstruction while meeting delay constraints. Fourth, they will apply these technologies to the coding and delivery of tiles-based 360-degree video by exploiting coding tools found in the new versatile video coding (VVC)/H.266 standard. Finally, they will propose new deep-learning approaches to evaluate the quality of damaged and repaired videos. The advances provided by this research will significantly improve the quality and usability of interactive and immersive video services, as well as other delay-sensitive applications, and enable the emergence of unprecedented media experiences. As for their previous research, Canada will benefit strongly from the intellectual property and commercialization of the technologies therein and from the training of highly qualified personnel in the fast-growing field of immersive video.