Mediated Uncoupled Learning: Learning Functions without Direct Input-output Correspondences
Mediated Uncoupled Learning: Learning Functions without Direct Input-output Correspondences
复制标题
DOI：
--
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Ikko Yamane;J. Honda;F. Yger;Masashi Sugiyama
中科院分区：
文献类型：
--
作者：
Ikko Yamane;J. Honda;F. Yger;Masashi Sugiyama
当我们将输入$X$和输出$Y$的训练数据配对时，普通监督学习是有用的。然而，这种配对数据在实践中可能很难收集。在本文中，我们考虑了当我们没有成对的数据时从$X$预测$Y$的任务，但我们有两个独立的独立的数据集$X$和$Y$，每个数据集都有某个中介变量$U$，即我们有两个数据集$S_X=(X_i，U_i)$和$S_Y=\{(U‘_j，Y’_j)\}$。一种天真的方法是使用$S_X$从$X$预测$U$，然后使用$S_Y$从$U$预测$Y$，但我们表明这在统计上并不一致。此外，在实践中，预测$U$可能比预测$Y$更困难，例如，当$U$具有更高的维度时。为了规避这一困难，我们提出了一种避免预测$U$而直接学习$Y=f(X)$的方法，即用$S_{X}$训练$f(X)$来预测经过$S_{Y}$训练的$h(U)$以逼近$Y$。我们证明了该方法的统计一致性和误差界，并通过实验验证了该方法的实用性。
Ordinary supervised learning is useful when we have paired training data of input $X$ and output $Y$. However, such paired data can be difficult to collect in practice. In this paper, we consider the task of predicting $Y$ from $X$ when we have no paired data of them, but we have two separate, independent datasets of $X$ and $Y$ each observed with some mediating variable $U$, that is, we have two datasets $S_X = \{(X_i, U_i)\}$ and $S_Y = \{(U'_j, Y'_j)\}$. A naive approach is to predict $U$ from $X$ using $S_X$ and then $Y$ from $U$ using $S_Y$, but we show that this is not statistically consistent. Moreover, predicting $U$ can be more difficult than predicting $Y$ in practice, e.g., when $U$ has higher dimensionality. To circumvent the difficulty, we propose a new method that avoids predicting $U$ but directly learns $Y = f(X)$ by training $f(X)$ with $S_{X}$ to predict $h(U)$ which is trained with $S_{Y}$ to approximate $Y$. We prove statistical consistency and error bounds of our method and experimentally confirm its practical usefulness.