Image Separation With Side Information: A Connected Auto-Encoders Based Approach.
Image Separation With Side Information: A Connected Auto-Encoders Based Approach.
复制标题
带有侧面信息的图像分离：基于连接自动编码器的方法。
DOI：
10.1109/tip.2023.3275872
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Pu W
中科院分区：
文献类型：
--
作者：
Pu W
X射线照相（X射线成像）是艺术研究中广泛使用的成像技术。它可以提供有关绘画状况的信息，以及对艺术家的技术和工作方法的见解，常常揭示肉眼看不见的隐藏信息。双面绘画的 X 射线照相会产生混合的 X 射线图像，本文处理分离该混合图像的问题。使用绘画每一面的可见彩色图像（RGB 图像），我们提出了一种基于“连接”自动编码器的新神经网络架构，旨在将混合 X 射线图像分离为对应于每一面的两个模拟 X 射线图像。这种连接的自动编码器架构使得编码器基于使用算法展开技术设计的卷积学习迭代收缩阈值算法（CLISTA），而解码器由简单的线性卷积层组成；编码器从前后绘画的可见图像和混合X射线图像中提取稀疏代码，而解码器则再现原始RGB图像和混合X射线图像。该学习算法以完全自我监督的方式运行，不需要包含混合 X 射线图像和分离图像的样本集。该方法在根特祭坛画双面翼板上的图像上进行了测试，该画由休伯特·凡·艾克 (Hubert van Eyck) 和扬·凡·艾克 (Jan van Eyck) 兄弟于 1432 年绘制。这些测试表明，所提出的方法优于用于艺术调查应用的其他最先进的 X 射线图像分离方法。
X-radiography (X-ray imaging) is a widely used imaging technique in art investigation. It can provide information about the condition of a painting as well as insights into an artist’s techniques and working methods, often revealing hidden information invisible to the naked eye. X-radiograpy of double-sided paintings results in a mixed X-ray image and this paper deals with the problem of separating this mixed image. Using the visible color images (RGB images) from each side of the painting, we propose a new Neural Network architecture, based upon ‘connected’ auto-encoders, designed to separate the mixed X-ray image into two simulated X-ray images corresponding to each side. This connected auto-encoders architecture is such that the encoders are based on convolutional learned iterative shrinkage thresholding algorithms (CLISTA) designed using algorithm unrolling techniques, whereas the decoders consist of simple linear convolutional layers; the encoders extract sparse codes from the visible image of the front and rear paintings and mixed X-ray image, whereas the decoders reproduce both the original RGB images and the mixed X-ray image. The learning algorithm operates in a totally self-supervised fashion without requiring a sample set that contains both the mixed X-ray images and the separated ones. The methodology was tested on images from the double-sided wing panels of the Ghent Altarpiece, painted in 1432 by the brothers Hubert and Jan van Eyck. These tests show that the proposed approach outperforms other state-of-the-art X-ray image separation methods for art investigation applications.