Losless and lossy compression of screen-content data using machine learning

该项目探索了一种新的屏幕内容数据压缩方法，该方法考虑了图像属性的不同方面和特定的压缩目标。与经典的照片或广播视频数据相比，屏幕内容图像的内容在统计属性方面非常不同。通常，图像中的某些区域具有两个典型属性：有限的颜色和重复的图案。研究表明，即使使用特殊工具，传统的压缩方法也不能有效地存储屏幕内容数据。基于理想数据编码的方法要成功得多。这种新方法的成功依赖于像素符号概率分布的优化建模。对于这种新的压缩方法，已经实现了一种概念性的方法(原型)，但它仍然局限于无损压缩，并且只对具有特定属性的图像内容获得高压缩效率。该项目研究了使用机器学习方法来建模像素符号概率分布的新方法。总体而言，该项目采用了几种方法：·现有的原型在一些处理步骤中只使用全局图像信息来建模概率分布。包含更多先验知识的估计方法，例如更具局部性质的估计方法，有望在压缩方面产生显著的收益。为此目的，正在考虑其他学习方法。总的来说，它是关于通过几个例子学习的。对于这种方法，可以获得比传统方法更高的压缩比，传统方法首先将图像分割为不同类型的区域，然后在必要时切换到传统压缩方法。·该研究项目还将研究合适的率失真优化如何将该过程扩展到几乎无损或有损压缩。专门的图像分析可以将优化参数化，以便考虑人类视觉的感知模型。·扩展图像序列压缩的概念性方法原则上是可能的，目的是。对时间部分的考虑可以改进对取决于情况的概率分布的建模，因此必须进行研究。在图像序列中，信号统计信息的变化通常是由于新的内容或场景改变而发生的。特别是，研究项目应该调查如何在现有的学习程序中补充适当的遗忘或重新学习的因素。

This project explores a novel approach to the compression of screen-content data, taking into account different facets of image properties and the specific objective of compression.In contrast to classical photo or broadcast-video data, the content of screen-content images is very diverse with regard to its statistical properties. Often, certain regions within the images are characterized by two typical properties: a limited number of colours and repeating patterns.Studies have shown that conventional compression methods are not able to store screen-content data efficiently even with the use of special tools. A method based on ideal data coding is much more successful. The success of this new method depends on an optimal modelling of the probability distributions of the pixel symbols.For this new compression method, a conceptual approach (prototype) has already been realized, which is however still limited to lossless compression and achieves a high compression efficiency only for image contents with certain properties.The project investigates new methods for modelling the probability distributions of pixel symbols using machine-learning methods. Overall, the project pursues several approaches:• The existing prototype only uses global image information in some processing steps for modelling the probability distributions. Estimation methods that incorporate more prior knowledge, e.g. of more local nature, are expected to yield significant gains in compression. Alternative learning methods are being considered for this purpose. In general, it is about learning with a few examples. For this approach, higher compression ratios can be achieved than with classical methods, which first segment the image into regions of different types and then switch to conventional compression methods if necessary.• The research project will also investigate how a suitable rate-distortion optimization can extend the procedure to a near-lossless or lossy compression. A dedicated image analysis can parameterize the optimization in order to consider perceptual models of human vision. • An extension of the conceptual approach to image sequence compression is possible in principle and is aimed at. The consideration of the temporal component can lead to an improved modelling of the probability distribution depending on the situation and must therefore be researched. In image sequences, changes in signal statistics typically occur due to new content or scene changes. In particular, the research project should investigate how the existing learning procedure can be supplemented with suitable elements for forgetting or relearning.