Robust contrastive learning and nonlinear ICA in the presence of outliers
Robust contrastive learning and nonlinear ICA in the presence of outliers
复制标题
DOI：
--
复制
发表时间：
2019-11
期刊：
影响因子：
--
通讯作者：
Hiroaki Sasaki;Takashi Takenouchi;R. Monti;Aapo Hyvärinen
中科院分区：
文献类型：
--
作者：
Hiroaki Sasaki;Takashi Takenouchi;R. Monti;Aapo Hyvärinen
非线性独立成分分析（ICA）是无监督表示学习的通用框架，旨在恢复数据中的潜在变量。最近的实用方法通过解决一系列基于逻辑回归的分类问题来执行非线性ICA。然而，众所周知，逻辑回归很容易受到异常值的影响，因此性能可能会被异常值严重削弱。在本文中，我们首先从理论上分析存在异常值的情况下的非线性 ICA 模型。我们的分析表明，当（未污染的）目标密度的尾部存在异常值时，非线性 ICA 中的估计可能会受到严重阻碍，这种情况发生在异常值污染的典型情况下。我们开发了两种基于 {\gamma}-散度的鲁棒非线性 ICA 方法，这是逻辑回归中 KL 散度的鲁棒替代方案。所提出的方法被证明在非线性 ICA 背景下具有所需的鲁棒性。我们还通过实验证明，所提出的方法非常稳健，并且在存在异常值的情况下优于现有方法。最后，所提出的方法应用于基于 ICA 的因果发现，并证明可以在 fMRI 数据上找到合理的因果关系。
Nonlinear independent component analysis (ICA) is a general framework for unsupervised representation learning, and aimed at recovering the latent variables in data. Recent practical methods perform nonlinear ICA by solving a series of classification problems based on logistic regression. However, it is well-known that logistic regression is vulnerable to outliers, and thus the performance can be strongly weakened by outliers. In this paper, we first theoretically analyze nonlinear ICA models in the presence of outliers. Our analysis implies that estimation in nonlinear ICA can be seriously hampered when outliers exist on the tails of the (noncontaminated) target density, which happens in a typical case of contamination by outliers. We develop two robust nonlinear ICA methods based on the {\gamma}-divergence, which is a robust alternative to the KL-divergence in logistic regression. The proposed methods are shown to have desired robustness properties in the context of nonlinear ICA. We also experimentally demonstrate that the proposed methods are very robust and outperform existing methods in the presence of outliers. Finally, the proposed method is applied to ICA-based causal discovery and shown to find a plausible causal relationship on fMRI data.