Modality Conversion of Handwritten Patterns by Cross Variational Autoencoders
Modality Conversion of Handwritten Patterns by Cross Variational Autoencoders
复制标题
DOI：
10.1109/icdar.2019.00072
复制
发表时间：
2019-06
期刊：
影响因子：
--
通讯作者：
Taichi Sumi;Brian Kenji Iwana;Hideaki Hayashi;S. Uchida
中科院分区：
文献类型：
--
作者：
Taichi Sumi;Brian Kenji Iwana;Hideaki Hayashi;S. Uchida
本研究试图建构一个网路，将线上与线下手写文字互相转换。该网络由两个可变自动编码器（VAE）组成，具有共享的潜在空间。VAE被训练成同时生成在线和离线手写拉丁字符。通过这种方式，我们创建了一个跨模态VAE（Cross-VAE）。在训练期间，所提出的交叉VAE被训练以最小化两种模态的重建损失、两个VAE的分布损失以及称为空间共享损失的新颖的第三损失。第三，空间共享损失用于通过计算潜变量之间的距离来鼓励模态共享相同的潜空间。通过该方法，在线和离线手写字符的相互转换是可能的。在本文中，我们证明了交叉VAE的性能，通过定性和定量分析。
This research attempts to construct a network that can convert online and offline handwritten characters to each other. The proposed network consists of two Variational Auto-Encoders (VAEs) with a shared latent space. The VAEs are trained to generate online and offline handwritten Latin characters simultaneously. In this way, we create a cross-modal VAE (Cross-VAE). During training, the proposed Cross-VAE is trained to minimize the reconstruction loss of the two modalities, the distribution loss of the two VAEs, and a novel third loss called the space sharing loss. This third, space sharing loss is used to encourage the modalities to share the same latent space by calculating the distance between the latent variables. Through the proposed method mutual conversion of online and offline handwritten characters is possible. In this paper, we demonstrate the performance of the Cross-VAE through qualitative and quantitative analysis.