A Lightweight Transformer Network for Hyperspectral Image Classification
A Lightweight Transformer Network for Hyperspectral Image Classification
复制标题
DOI：
10.1109/tgrs.2023.3297858
复制
发表时间：
2023
影响因子：
8.2
通讯作者：
Xuming Zhang;Yuanchao Su;Lianru Gao;L. Bruzzone;Xingfa Gu;Q. Tian
中科院分区：
文献类型：
--
作者：
Xuming Zhang;Yuanchao Su;Lianru Gao;L. Bruzzone;Xingfa Gu;Q. Tian
变换是获取长距离相关性的有力工具，在高光谱图像分类中表现出了令人印象深刻的性能。然而，这样的能力伴随着巨大的内存占用和巨大的计算负担。在本文中，我们提出了两种类型的轻量级自我注意模块(通道轻量级多头自我注意模块(CLMSA)和位置轻量级多头自我注意模块(PLMSA))，以减少存储和计算，同时将每个像素或通道与全局信息相关联。此外，我们发现，由于固定的输入大小，变换不能有效地显式地提取局部和多尺度特征，并且在处理少量训练样本时容易过度拟合。因此，提出了一种使用所提出的轻量级自我注意模块构建的轻量级变压器(LIT)网络。LIT采用卷积块来显式地提取早期层的局部信息，并使用转换器来捕获深层的远程依赖关系。此外，我们设计了一种受控多类分层(CMS)采样策略，以生成适当大小的输入数据，确保均衡采样，并减少训练样本和测试样本之间特征提取区域的重叠。通过适当的训练数据、卷积标记化和LITS，LIT减轻了过拟合，同时具有高计算效率和良好的性能。在几个HSI数据集上的实验结果验证了我们的设计的有效性。
Transformer is a powerful tool for capturing long-range dependencies and has shown impressive performance in hyperspectral image (HSI) classification. However, such power comes with a heavy memory footprint and huge computation burden. In this article, we propose two types of lightweight self-attention modules (a channel lightweight multihead self-attention (CLMSA) module and a position lightweight multihead self-attention (PLMSA) module) to reduce both memory and computation while associating each pixel or channel with global information. Moreover, we discover that transformers are ineffective in explicitly extracting local and multiscale features due to the fixed input size and tend to overfit when dealing with a small number of training samples. Therefore, a lightweight transformer (LiT) network, built with the proposed lightweight self-attention modules, is presented. LiT adopts convolutional blocks to explicitly extract local information in early layers and employs transformers to capture long-range dependencies in deep layers. Furthermore, we design a controlled multiclass stratified (CMS) sampling strategy to generate appropriately sized input data, ensure balanced sampling, and reduce the overlap of feature extraction regions between training and test samples. With appropriate training data, convolutional tokenization, and LiTs, LiT mitigates overfitting and enjoys both high computational efficiency and good performance. Experimental results on several HSI datasets verify the effectiveness of our design.