Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
复制标题
DOI：
--
复制
发表时间：
2016-05
期刊：
影响因子：
--
通讯作者：
Aapo Hyvärinen;H. Morioka
中科院分区：
文献类型：
--
作者：
Aapo Hyvärinen;H. Morioka
非线性独立成分分析（ICA）为无监督特征学习提供了一个有吸引力的框架，但迄今为止提出的模型还无法识别。在这里，我们首先提出了一种利用数据非平稳结构的时间序列无监督深度学习的新直观原理。我们的学习原理，时间对比学习（TCL），找到了一种允许最佳区分时间段（窗口）的表示。令人惊讶的是，我们展示了当重新定义 ICA 以包含时间非平稳性时，TCL 如何与非线性 ICA 模型相关。特别是，我们表明，TCL 与线性 ICA 相结合，估计非线性 ICA 模型直至源的逐点变换，并且该解决方案是独特的——从而为非线性 ICA 提供了第一个可辨识的结果，该结果是严格的、建设性的且非常通用的。
Nonlinear independent component analysis (ICA) provides an appealing framework for unsupervised feature learning, but the models proposed so far are not identifiable. Here, we first propose a new intuitive principle of unsupervised deep learning from time series which uses the nonstationary structure of the data. Our learning principle, time-contrastive learning (TCL), finds a representation which allows optimal discrimination of time segments (windows). Surprisingly, we show how TCL can be related to a nonlinear ICA model, when ICA is redefined to include temporal nonstationarities. In particular, we show that TCL combined with linear ICA estimates the nonlinear ICA model up to point-wise transformations of the sources, and this solution is unique --- thus providing the first identifiability result for nonlinear ICA which is rigorous, constructive, as well as very general.