Augmented Equivariant Attention Networks for Microscopy Image Transformation
Augmented Equivariant Attention Networks for Microscopy Image Transformation
复制标题
用于显微镜图像转换的增强等变注意网络
DOI：
10.1109/tmi.2022.3179665
复制
发表时间：
2022
影响因子：
10.6
通讯作者：
Ji, Shuiwang
中科院分区：
文献类型：
--
作者：
Xie, Yaochen;Ding, Yu;Ji, Shuiwang
拍摄高质量或高分辨率的电子显微镜（EM）和荧光显微镜（FM）图像耗时且昂贵。拍摄这些图像甚至可能会对样品造成伤害，并且可能会在长时间或高强度曝光后破坏样品中的某些细微之处，这通常是首先实现高质量或高分辨率所必需的。深度学习的进步使我们能够执行各种类型的显微图像到图像转换任务，例如图像去噪、超分辨率和分割，这些任务可以从物理获取的低质量图像中计算生成高质量图像。在实验获得的显微镜图像对上训练图像到图像转换模型时，先前的模型由于无法捕获图像间的依赖关系和图像之间共享的共同特征而遭受性能损失。现有的利用图像分类任务中的共享特征的方法不能很好地应用于图像变换任务，因为它们不能保持图像到图像变换中必不可少的空间置换的等方差特性。为了解决这些限制，我们提出了增强等变注意网络（AEANets），它具有更好的捕获图像间依赖关系的能力，同时保持了等变特性。本文提出的AEANets通过对注意力机制的两种增强来捕获图像间的依赖关系和共享特征，即在训练过程中共享引用和批感知注意力。我们从理论上推导了所提出的增强注意模型的等方差特性，并通过实验证明了其在定量和视觉结果上优于基线方法的一致性。
It is time-consuming and expensive to take high-quality or high-resolution electron microscopy (EM) and fluorescence microscopy (FM) images. Taking these images could be even invasive to samples and may damage certain subtleties in the samples after long or intense exposures, often necessary for achieving high-quality or high-resolution in the first place. Advances in deep learning enable us to perform various types of microscopy image-to-image transformation tasks such as image denoising, super-resolution, and segmentation that computationally produce high-quality images from the physically acquired low-quality ones. When training image-to-image transformation models on pairs of experimentally acquired microscopy images, prior models suffer from performance loss due to their inability to capture inter-image dependencies and common features shared among images. Existing methods that take advantage of shared features in image classification tasks cannot be properly applied to image transformation tasks because they fail to preserve the equivariance property under spatial permutations, something essential in image-to-image transformation. To address these limitations, we propose the augmented equivariant attention networks (AEANets) with better capability to capture inter-image dependencies, while preserving the equivariance property. The proposed AEANets captures inter-image dependencies and shared features via two augmentations on the attention mechanism, which are the shared references and the batch-aware attention during training. We theoretically derive the equivariance property of the proposed augmented attention model and experimentally demonstrate its consistent superiority in both quantitative and visual results over the baseline methods.
深度学习中注意力模型的数学观点
DOI：
--
发表时间：
2020
期刊：
影响因子：
--
作者：
Shuiwang Ji;Yaochen Xie
通讯作者：
Yaochen Xie
影响因子：
23.8
作者：
Zhengyang Wang;Yaochen Xie;Shuiwang Ji
通讯作者：
Zhengyang Wang;Yaochen Xie;Shuiwang Ji
影响因子：
48
作者：
Wu, Yichen;Rivenson, Yair;Ozcan, Aydogan
通讯作者：
Ozcan, Aydogan