Joint Camera Spectral Response Selection and Hyperspectral Image Recovery
Joint Camera Spectral Response Selection and Hyperspectral Image Recovery
复制标题
联合相机光谱响应选择和高光谱图像恢复
DOI：
10.1109/tpami.2020.3009999
复制
发表时间：
2022-01-01
影响因子：
23.6
通讯作者：
Huang, Hua
中科院分区：
文献类型：
--
作者：
Fu, Ying;Zhang, Tao;Huang, Hua
从一幅RGB图像中恢复高光谱图像已经引起了人们的广泛关注，其性能对相机的光谱响应(CSR)非常敏感。本文提出了一种高效的基于卷积神经网络(CNN)的方法，该方法可以从候选数据集中联合选择最优的CSR，并学习映射以从多芯片或单芯片设置下使用该算法选择的相机捕获的单个RGB图像恢复HSI。在给定特定的CSR的情况下，我们首先提出了HSI恢复网络，该网络解释了HSI的基本特征，包括频谱非线性映射和空间相似性。随后，我们在恢复网络中加入CSR选择层，从而在非负稀疏约束下根据网络权重自动确定多芯片和单芯片设置下的最优CSR。在三个高光谱数据集和两个相机光谱响应数据集上的实验结果表明，我们的HSI恢复网络在量化指标和感知质量方面都优于最先进的方法，并且选择层总是返回与穷举搜索确定的最佳CSR一致的CSR。最后，我们证明了我们的方法在实际捕获系统中也能取得很好的效果，并收集了一个高光谱花卉数据集来评估HSI恢复对分类问题的影响。
Hyperspectral image (HSI) recovery from a single RGB image has attracted much attention, whose performance has recently been shown to be sensitive to the camera spectral response (CSR). In this paper, we present an efficient convolutional neural network (CNN) based method, which can jointly select the optimal CSR from a candidate dataset and learn a mapping to recover HSI from a single RGB image captured with this algorithmically selected camera under multi-chip or single-chip setups. Given a specific CSR, we first present a HSI recovery network, which accounts for the underlying characteristics of the HSI, including spectral nonlinear mapping and spatial similarity. Later, we append a CSR selection layer onto the recovery network, and the optimal CSR under both multi-chip and single-chip setups can thus be automatically determined from the network weights under the nonnegative sparse constraint. Experimental results on three hyperspectral datasets and two camera spectral response datasets demonstrate that our HSI recovery network outperforms state-of-the-art methods in terms of both quantitative metrics and perceptive quality, and the selection layer always returns a CSR consistent to the best one determined by exhaustive search. Finally, we show that our method can also perform well in the real capture system, and collect a hyperspectral flower dataset to evaluate the effect from HSI recovery on classification problem.