Contour-based Multidirectional Prediction for Intra Coding

内部编码是所有现代图像和视频编解码器的重要组成部分。它用于传输的开始，用于随机访问正在进行的传输，用于错误恢复，用于改变视频点播应用中的数据速率以及用于视频序列中新内容的编码。与间编码不同，内编码不使用任何先前传输的信息，从而提高了编码效率。因此，在相同的图像质量下，帧内编码的数据速率是帧间编码的10到100倍。这就是为什么我们需要在内编码的空间预测领域进行进一步的研究。已知的内编码预测方法在信号轮廓丰富的情况下表现不佳。这在多个或非线性轮廓的情况下尤其正确。因此，发展了包含多个轮廓线的信号的预测方法。根据文献和我们自己的工作，我们知道对轮廓的外推和像素值的预测有不同的方法是有用的。没有一种已知的方法可以提供一个模型来描述线性和非线性轮廓，以及方向变化是可预测的轮廓。然而，为了最小化数据速率，需要一个通用模型来正确预测这些复杂的轮廓。已知的预测像素值的方法不够精确，过于计算密集或不适合作为未来图像帧的预测信号。该项目的目标是通过将传统的视频编码方法与轮廓外推和像素值预测的机器学习方法相结合，开发更有效的图像编码和视频编码内编码方法。我们将使用高斯过程在先前编码的相邻块中建模轮廓。该模型将适用于任何轮廓，无论是线性，非线性或方向变化。使用高斯过程，我们将轮廓外推到接下来要编码的图像区域。使用之前编码的图像信号和外推的轮廓作为参考，我们将使用CNN来预测像素值。将这些方法集成到图像和视频编解码器中，可以提高图像和视频编码的编码效率。

Intra coding is an essential part of all modern image and video codecs. It is used at the beginning of a transmission, for random access into ongoing transmissions, for error resilience, for changing the data rate in video on demand applications as well as for coding of new content in video sequences. Different to inter coding, intra coding does not use any previously transmitted information for improving the coding efficiency. Therefore, the data rate for intra coding is 10-fold to 100-fold compared to inter coding for the same picture quality. This is the reason for conducting further research in the area of spatial prediction for intra coding.Known predictions methods for intra coding do not perform well in case of a signal rich in contours. This is especially true in case of several or non-linear contours. Therefore methods for the prediction of signals containing several contours were developed. Based on the literature and our own work it is known that it is useful to have separate methods for the extrapolation of contours and for the prediction of pixel values. None of the known methods provides a model for contours that can describe linear and non-linear contours as well as contours where the change of direction is predictable. However, a generic model is desirable for proper prediction of these complex contours in order to minimize the data rate. Known methods for the prediction of pixel values are not sufficiently precise, too compute intensive or unsuitable as a prediction signal for future image frames.The goal of this project is the development of more efficient intra coding methods for image coding as well as video coding by combining traditional video coding methods with contour extrapolation and machine learning methods for pixel value prediction. We will model contours in previously coded neighboring blocks using a Gauss process. The model will be appropriate for any contour, be it linear, non-linear or direction changing. Using the Gauss process, we will extrapolate the contours into the areas of the image to be coded next. Using the previously coded image signal and the extrapolated contours as a reference, we will use a CNN to predict the pixel values. The integration of these methods into image and video codecs will show that the coding efficiency of image and video coding can be improved.