Harmony: A Generic Unsupervised Approach for Disentangling Semantic Content from Parameterized Transformations.
Harmony: A Generic Unsupervised Approach for Disentangling Semantic Content from Parameterized Transformations.
复制标题
DOI：
10.1109/cvpr52688.2022.01999
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Xu, Min
中科院分区：
文献类型：
--
作者：
Uddin, Mostofa Rafid;Howe, Gregory;Zeng, Xiangrui;Xu, Min
在许多现实生活中的图像分析应用中，特别是在生物医学研究领域，感兴趣的对象经历多次变换，改变其视觉属性，同时保持语义内容不变。将图像分解为语义内容因素和转换可以为许多特定领域的图像分析任务提供显著的好处。为此，我们提出了一个通用的无监督框架Harmony，它可以同时明确地从多个参数化转换中分离语义内容。Harmony利用一个简单的交叉对比学习框架和多个明确参数化的潜在表征来从转换中分离内容。为了证明Harmony的有效性，我们将其应用于从几个参数化转换（旋转、平移、缩放和对比度）中分离图像语义内容。Harmony在不同领域的多个图像数据集上实现了对基线模型的显著改进。通过这种解纠缠，Harmony被证明可以通过从冷冻电镜图像中建模大分子的结构异质性和从单颗粒冷冻电镜图像中学习蛋白质颗粒的变换不变表示来激励生物图像分析研究。Harmony在从3D变换中解纠缠内容方面也表现得非常好，并且可以执行3D cryo-ET子层析图的粗对齐和快速对齐。因此，Harmony可以推广到许多其他成像领域，也可以潜在地扩展到成像以外的领域。
In many real-life image analysis applications, particularly in biomedical research domains, the objects of interest undergo multiple transformations that alters their visual properties while keeping the semantic content unchanged. Disentangling images into semantic content factors and transformations can provide significant benefits into many domain-specific image analysis tasks. To this end, we propose a generic unsupervised framework, Harmony, that simultaneously and explicitly disentangles semantic content from multiple parameterized transformations. Harmony leverages a simple cross-contrastive learning framework with multiple explicitly parameterized latent representations to disentangle content from transformations. To demonstrate the efficacy of Harmony, we apply it to disentangle image semantic content from several parameterized transformations (rotation, translation, scaling, and contrast). Harmony achieves significantly improved disentanglement over the baseline models on several image datasets of diverse domains. With such disentanglement, Harmony is demonstrated to incentivize bioimage analysis research by modeling structural heterogeneity of macromolecules from cryo-ET images and learning transformation-invariant representations of protein particles from single-particle cryo-EM images. Harmony also performs very well in disentangling content from 3D transformations and can perform coarse and fast alignment of 3D cryo-ET subtomograms. Therefore, Harmony is generalizable to many other imaging domains and can potentially be extended to domains beyond imaging as well.
登录
查看更多内容
影响因子：
5.6
作者：
Harastani, Mohamad;Eltsov, Mikhail;Jonic, Slavica
通讯作者：
Jonic, Slavica
影响因子：
7.8
作者：
Hyvärinen, A;Oja, E
通讯作者：
Oja, E
影响因子：
3.2
作者：
TURK, M;PENTLAND, A
通讯作者：
PENTLAND, A
影响因子：
64.5
作者：
Guo Q;Lehmer C;Martínez-Sánchez A;Rudack T;Beck F;Hartmann H;Pérez-Berlanga M;Frottin F;Hipp MS;Hartl FU;Edbauer D;Baumeister W;Fernández-Busnadiego R
通讯作者：
Fernández-Busnadiego R
影响因子：
9.9
作者：
Petersen, R. C.;Aisen, P. S.;Weiner, M. W.
通讯作者：
Weiner, M. W.