Low-delay reliable video delivery for real-time immersive applications

近年来，视频设备、应用程序和系统的快速部署，这种增长只会加剧。大量视频数据将在非常创新的应用中传输，例如实时沉浸式视频（例如，360度视频）、视觉远程机械控制、远程手术和实时车际通信。在异构网络上部署了各种各样的交互式视频应用程序，确保视频数据的可靠传输，特别是在低延迟约束下，是一项极具挑战性的工作。具体来说，无线网络上的传输错误和数据包丢失会导致显著的延迟和视觉质量下降。因此，在寻求为最终用户提供最佳质量的体验时，以快速可靠的方式传输视频内容的能力构成了5G等现代网络最关键的挑战之一。本研究计划的主要目的是提出新的解决方案，以确保在沉浸式视频应用的背景下，在非常低的延迟约束下，在异构网络上可靠地传输视频数据。具体来说，申请人和他的团队将创造有前途的方法来保护视频数据免受丢失和传输错误，以及纠正和修复视频内容的方法。这将消除重传的需要，重传对于非常低延迟的应用程序是不希望的或不适合的。在这一探索中，他们将追求五个研究方向。首先，他们将创建适合于低延迟交互式视频应用的新型前向纠错（FEC）方法。其次，他们将提出一种全新的方法，结合他们新开发的纠错视频数据包的算法，使用循环冗余检查信息，与这些新的FEC类，以保护系统免受比特错误和数据包丢失。第三，他们将开发新的机器学习方法来动态地调整FEC参数以适应视频内容和不断变化的网络条件，以确保在满足延迟约束的同时无错误地重建视频。第四，他们将利用新的多功能视频编码(VVC)/H中的编码工具，将这些技术应用于基于瓦片的360度视频的编码和传输。266标准。最后，他们将提出新的深度学习方法来评估损坏和修复的视频的质量。本研究提供的进展将显著提高交互式和沉浸式视频服务的质量和可用性，以及其他延迟敏感应用，并使前所未有的媒体体验出现。至于他们之前的研究，加拿大将从其中的技术的知识产权和商业化以及在快速发展的沉浸式视频领域培养高素质人才中受益匪浅。

Recent years have witnessed a rapid deployment of video devices, applications and systems, and this growth will only intensify. Large amounts of video data will be streamed in very innovative applications such as live immersive video (e.g., 360-degree video), visual remote machinery control, remote surgery and live inter-vehicular communications. A wide diversity of interactive video applications are deployed on heterogeneous networks, and ensuring the reliable delivery of video data, especially under low delay constraints, is highly challenging. Specifically, transmission errors and packet losses over wireless networks cause significant delays and visual quality degradation. Thus, in seeking to provide the best quality of experience to end users, the ability to transport video content in a fast and reliable manner constitutes one of the most crucial challenges for modern networks, such as 5G. This research program aims mainly to propose novel solutions to the problem of ensuring the reliable delivery of video data over heterogeneous networks, under very low delay constraints, in the context of immersive video applications. Specifically, the applicant and his team will create promising methods to protect video data against losses and transmission errors, in addition to methods to correct and repair video content. These will remove the need for retransmissions, which are undesirable or unsuitable for very low-delay applications. In this quest, they will pursue five research tracks. First, they will create new classes of forward error correction (FEC) methods suitable for low-delay interactive video applications. Second, they will propose a radically new approach combining their newly developed algorithms for correcting erroneous video packets, using the Cyclic Redundancy Check information, with these new FEC classes, to protect the system against both bit errors and packets losses. Third, they will develop new machine learning approaches to dynamically adapt the FEC parameters to the video content and ever-changing network conditions in order to ensure error-free video reconstruction while meeting delay constraints. Fourth, they will apply these technologies to the coding and delivery of tiles-based 360-degree video by exploiting coding tools found in the new versatile video coding (VVC)/H.266 standard. Finally, they will propose new deep-learning approaches to evaluate the quality of damaged and repaired videos. The advances provided by this research will significantly improve the quality and usability of interactive and immersive video services, as well as other delay-sensitive applications, and enable the emergence of unprecedented media experiences. As for their previous research, Canada will benefit strongly from the intellectual property and commercialization of the technologies therein and from the training of highly qualified personnel in the fast-growing field of immersive video.