CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction.
CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction.
复制标题
DOI：
10.3390/e24040456
复制
发表时间：
2022-03-25
期刊：
影响因子：
2.7
通讯作者：
Ma, Yi
中科院分区：
文献类型：
--
作者：
Dai, Xili;Tong, Shengbang;Li, Mingyang;Wu, Ziyang;Psenka, Michael;Chan, Kwan Ho Ryan;Zhai, Pengyuan;Yu, Yaodong;Yuan, Xiaojun;Shum, Heung-Yeung;Ma, Yi
这项工作提出了一个新的计算框架，用于学习现实世界数据集的结构化生成模型。特别是，我们建议在由多个独立的多维线性子空间组成的特征空间中学习多类、多维数据分布和线性判别表示（CTRL）之间的闭环转录。特别地，我们认为所寻求的最优编码和解码映射可以被表述为编码器和解码器之间的最小最大二人博弈。这个游戏的一个自然效用函数是所谓的率降低，这是一个简单的信息论度量，用于特征空间中类子空间高斯分布混合之间的距离。我们的公式从控制系统的闭环误差反馈中获得灵感，避免了昂贵的评估和最小化数据空间或特征空间中任意分布之间的近似距离。在很大程度上，这个新公式统一了自动编码和GAN的概念和优点，并自然地将它们扩展到学习多类和多维现实世界数据的判别和生成表示的设置。我们在许多基准图像数据集上的大量实验证明了这种新的闭环公式的巨大潜力：在公平的比较下，学习的解码器的视觉质量和编码器的分类性能是有竞争力的，可以说比基于GAN， VAE或两者结合的现有方法更好。与现有的生成模型不同，该模型将多类的已知特征结构化而不是隐藏，将不同的类显式映射到特征空间中相应的独立主子空间，并通过每个子空间中的独立主成分来建模每个类中的不同视觉属性。
This work proposes a new computational framework for learning a structured generative model for real-world datasets. In particular, we propose to learn a Closed-loop Transcriptionbetween a multi-class, multi-dimensional data distribution and a Linear discriminative representation (CTRL) in the feature space that consists of multiple independent multi-dimensional linear subspaces. In particular, we argue that the optimal encoding and decoding mappings sought can be formulated as a two-player minimax game between the encoder and decoderfor the learned representation. A natural utility function for this game is the so-called rate reduction, a simple information-theoretic measure for distances between mixtures of subspace-like Gaussians in the feature space. Our formulation draws inspiration from closed-loop error feedback from control systems and avoids expensive evaluating and minimizing of approximated distances between arbitrary distributions in either the data space or the feature space. To a large extent, this new formulation unifies the concepts and benefits of Auto-Encoding and GAN and naturally extends them to the settings of learning a both discriminative and generative representation for multi-class and multi-dimensional real-world data. Our extensive experiments on many benchmark imagery datasets demonstrate tremendous potential of this new closed-loop formulation: under fair comparison, visual quality of the learned decoder and classification performance of the encoder is competitive and arguably better than existing methods based on GAN, VAE, or a combination of both. Unlike existing generative models, the so-learned features of the multiple classes are structured instead of hidden: different classes are explicitly mapped onto corresponding independent principal subspaces in the feature space, and diverse visual attributes within each class are modeled by the independent principal components within each subspace.
登录
查看更多内容
影响因子：
3.7
作者：
HASTIE, T;STUETZLE, W
通讯作者：
STUETZLE, W
DOI：
10.1090/s0002-9939-10-10340-2
发表时间：
2010-07-01
影响因子：
1
作者：
Bacharoglou, Athanassia G.
通讯作者：
Bacharoglou, Athanassia G.
影响因子：
2.2
作者：
Bennett, Jonathan;Carbery, Anthony;Tao, Terence
通讯作者：
Tao, Terence
影响因子：
3.6
作者：
El Ghaoui, Laurent;Gu, Fangda;Tsai, Alicia
通讯作者：
Tsai, Alicia