Machine Learning based Video Coding Optimizations: A Survey
Machine Learning based Video Coding Optimizations: A Survey
复制标题
基于机器学习的视频编码优化：调查
DOI：
10.1016/j.ins.2019.07.096
复制
发表时间：
2020
影响因子：
8.1
通讯作者：
Shiqi Wang
中科院分区：
文献类型：
--
作者：
Yun Zhang;Sam Kwong;Shiqi Wang
视频数据已成为全球消费的最大数据来源。由于视频应用的快速增长和对更高质量视频服务的需求不断增加，全球范围内视频数据量呈爆炸式增长，这对多媒体计算、传输和存储提出了最严峻的挑战。通过将视频压缩成更小的尺寸来进行视频编码是关键解决方案之一；然而，近三十年来，随着压缩比的不断增长，其发展已达到一定程度的饱和。机器学习算法，特别是深度学习算法，能够从非结构化海量数据中发现知识并提供数据驱动的预测，为视频编码技术的进一步升级提供了新的机遇。在本文中，我们对基于机器学习的视频编码优化进行了综述，旨在为研究人员提供坚实的基础并激发数据驱动视频编码的未来发展。首先，我们分析视频数据的表示和冗余。其次，我们回顾了视频编码标准的发展和关键要求。随后，我们从高效率、低复杂度和高视觉质量三个关键方面对基于机器学习的视频编码优化的最新进展和挑战进行了系统调查。详细分析了它们的工作流程、代表性方案、性能、优缺点。最后，确定了挑战和机遇，这可能为学术界和工业界提供未来研究的基础和潜在方向。
Video data has become the largest source of data consumed globally. Due to the rapid growth of video applications and boosting demands for higher quality video services, video data volume has been increasing explosively worldwide, which has been the most severe challenge for multimedia computing, transmission and storage. Video coding by compress- ing videos into a much smaller size is one of the key solutions; however, its development has become saturated to some extent while the compression ratio continuously grows in the last three decades. Machine leaning algorithms, especially those employing deep learning, which are capable of discovering knowledge from unstructured massive data and providing data-driven predictions, provide new opportunities for further upgrading video coding technologies. In this article, we present a review on machine learning based video encoding optimization, aiming to provide researchers with a strong foundation and inspire future developments for data-driven video coding. Firstly, we analyze the representations and redundancies of video data. Secondly, we review the development of video coding standards and key requirements. Subsequently, we present a systemic survey on the re- cent advances and challenges associated with the machine learning based video coding optimizations from three key aspects, including high efficiency, low complexity and high visual quality. Their workflows, representative schemes, performances, advantages and dis- advantages are analyzed in detail. Finally, the challenges and opportunities are identified, which may provide the academic and industrial communities with groundwork and poten- tial directions for future research.