Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators
Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators
复制标题
DOI：
--
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Takeshi Teshima;Isao Ishikawa;Koichi Tojo;Kenta Oono;M. Ikeda;Masashi Sugiyama
中科院分区：
文献类型：
--
作者：
Takeshi Teshima;Isao Ishikawa;Koichi Tojo;Kenta Oono;M. Ikeda;Masashi Sugiyama
基于耦合流的可逆神经网络（CF-INN）具有各种机器学习应用，例如图像合成和表示学习。然而，他们的理想特性，如分析可逆性，是以限制函数形式为代价的。这就提出了一个关于其表示能力的问题：CF-INN是可逆函数的通用逼近器吗？如果没有普适性，可能会有一个行为良好的可逆转换，CF-INN永远无法近似，因此它会使模型类不可靠。我们回答这个问题，显示一个方便的标准：CF-INN是普遍的，如果它的层包含仿射耦合和可逆的线性函数作为特殊情况。作为它的推论，我们可以肯定地解决以前未解决的问题：是否规范化流模型的基础上仿射耦合可以是通用的分布逼近。在证明的普遍性的过程中，我们证明了一个一般定理，以显示的普遍性的等价性，某些非同构类，一个理论的洞察力，是感兴趣的本身。
Invertible neural networks based on coupling flows (CF-INNs) have various machine learning applications such as image synthesis and representation learning. However, their desirable characteristics such as analytic invertibility come at the cost of restricting the functional forms. This poses a question on their representation power: are CF-INNs universal approximators for invertible functions? Without a universality, there could be a well-behaved invertible transformation that the CF-INN can never approximate, hence it would render the model class unreliable. We answer this question by showing a convenient criterion: a CF-INN is universal if its layers contain affine coupling and invertible linear functions as special cases. As its corollary, we can affirmatively resolve a previously unsolved problem: whether normalizing flow models based on affine coupling can be universal distributional approximators. In the course of proving the universality, we prove a general theorem to show the equivalence of the universality for certain diffeomorphism classes, a theoretical insight that is of interest by itself.