Covariate-informed Representation Learning to Prevent Posterior Collapse of iVAE
Covariate-informed Representation Learning to Prevent Posterior Collapse of iVAE
复制标题
DOI：
--
复制
发表时间：
2022-02
期刊：
影响因子：
--
通讯作者：
Young-geun Kim;Y. Liu;Xue Wei
中科院分区：
文献类型：
--
作者：
Young-geun Kim;Y. Liu;Xue Wei
最近提出的可识别变分自动编码器（iVAE）框架提供了一个很有前途的方法来学习潜在的独立成分（IC）。iVAE使用辅助协变量来构建从协变量到IC到观测值的可识别生成结构，并且后验网络近似给定观测值和协变量的IC。虽然可识别性是有吸引力的，但我们表明iVAE可能具有局部最小解，其中观测值和近似IC是独立的。我们将这种现象称为iVAE的后塌陷问题。为了克服这个问题，我们开发了一种新的方法，协变量知情的iVAE（CI-iVAE），通过考虑编码器和后验分布的目标函数的混合物。在这样做时，目标函数防止后验崩溃，从而产生包含更多观测信息的潜在表示。此外，CI-iVAE扩展了原来的iVAE目标函数到一个更大的类，并找到其中的最佳，从而有更严格的证据下限比原来的iVAE。在仿真数据集、EMNIST、Fashion-MNIST和大规模脑成像数据集上的实验证明了该方法的有效性。
The recently proposed identifiable variational autoencoder (iVAE) framework provides a promising approach for learning latent independent components (ICs). iVAEs use auxiliary covariates to build an identifiable generation structure from covariates to ICs to observations, and the posterior network approximates ICs given observations and covariates. Though the identifiability is appealing, we show that iVAEs could have local minimum solution where observations and the approximated ICs are independent given covariates.-a phenomenon we referred to as the posterior collapse problem of iVAEs. To overcome this problem, we develop a new approach, covariate-informed iVAE (CI-iVAE) by considering a mixture of encoder and posterior distributions in the objective function. In doing so, the objective function prevents the posterior collapse, resulting latent representations that contain more information of the observations. Furthermore, CI-iVAE extends the original iVAE objective function to a larger class and finds the optimal one among them, thus having tighter evidence lower bounds than the original iVAE. Experiments on simulation datasets, EMNIST, Fashion-MNIST, and a large-scale brain imaging dataset demonstrate the effectiveness of our new method.