RF-Inpainter: Multimodal Image Inpainting Based on Vision and Radio Signals
RF-Inpainter: Multimodal Image Inpainting Based on Vision and Radio Signals
复制标题
DOI：
10.1109/access.2022.3214972
复制
发表时间：
2022
期刊：
影响因子：
3.9
通讯作者：
Cheng Chen;T. Nishio;M. Bennis;Jihong Park
中科院分区：
文献类型：
--
作者：
Cheng Chen;T. Nishio;M. Bennis;Jihong Park
这项研究证明了图像修复使用视觉信息和射频（RF）信号的可行性。使用RF信号的成像和基于视觉的技术的最新发展揭示了利用多模态信息来增强图像修复性能的潜力。在这种情况下，我们提出了RF-Inpainter-一种新的修复方法，通过使用深度自动编码器模型将有缺陷的RGB图像与接收信号强度指示器（RSSI）融合，将视觉和无线信息整合在一起。使用室内环境中实验获得的图像和RSSI数据集来评估RF-Inpainter的修复性能。仅图像修复和仅RSSI修复模型被用作基线，以说明RF-Inpainter优于基于单一模态的修复方法。结果表明，RF-Inpainter在大多数实验场景中生成令人满意的修复图像，在平均峰值信噪比（PSNR）和平均结构相似性指数（SSIM）方面分别实现了36.4%和14.6%的最大改善。
This study demonstrates the feasibility of image inpainting using both visual information and radio frequency (RF) signals. Recent developments in imaging and vision-based technologies using RF signals have revealed the potential of leveraging multimodal information to enhance image inpainting performance. In this context, we propose RF-Inpainter—a novel inpainting method that integrates visual and wireless information by fusing defective RGB images with received signal strength indicator (RSSI) using a deep auto-encoder model. The inpainting performance of RF-Inpainter is evaluated using experimentally obtained images and RSSI datasets in an indoor environment. Image-only inpainting and RSSI-only inpainting models are used as baselines to illustrate the superiority of RF-Inpainter over inpainting methods based on a single modality. The results establish that RF-Inpainter generates satisfactory inpainted images in most experimental scenarios, achieving a maximum improvement of 36.4% and 14.6% in terms of mean peak signal-to-noise ratio (PSNR) and mean structural similarity index (SSIM), respectively.