On Learning Invariant Representations for Domain Adaptation
On Learning Invariant Representations for Domain Adaptation
复制标题
DOI：
--
复制
发表时间：
2019-05
影响因子：
2.4
通讯作者：
H. Zhao;Rémi Tachet des Combes;Kun Zhang;Geoffrey J. Gordon
中科院分区：
文献类型：
--
作者：
H. Zhao;Rémi Tachet des Combes;Kun Zhang;Geoffrey J. Gordon
由于深度神经网络学习丰富表示的能力，无监督域自适应的最新进展集中在学习域不变特征，这些特征在源域上实现小误差。希望学习的表示，连同从源域学习的假设，可以推广到目标域。在本文中，我们首先构建一个简单的反例表明，与常见的信念相反，上述条件不足以保证成功的域适应。特别是，反例表现出条件转移：输入特征的类条件分布在源域和目标域之间变化。为了给出域适应的充分条件，我们提出了一个自然的和可解释的泛化上限，明确考虑到上述转变。此外，我们揭示了新的光的问题，证明了信息理论的联合错误的任何域自适应方法，试图学习不变表示的下限。我们的结果表征了学习不变表示和实现两个域上的小联合误差时，从源到目标的边缘标签分布不同的基本权衡。最后，我们在真实世界的数据集上进行实验，证实了我们的理论发现。我们相信这些见解有助于指导未来的领域自适应和表示学习算法的设计。
Due to the ability of deep neural nets to learn rich representations, recent advances in unsupervised domain adaptation have focused on learning domain-invariant features that achieve a small error on the source domain. The hope is that the learnt representation, together with the hypothesis learnt from the source domain, can generalize to the target domain. In this paper, we first construct a simple counterexample showing that, contrary to common belief, the above conditions are not sufficient to guarantee successful domain adaptation. In particular, the counterexample exhibits conditional shift: the class-conditional distributions of input features change between source and target domains. To give a sufficient condition for domain adaptation, we propose a natural and interpretable generalization upper bound that explicitly takes into account the aforementioned shift. Moreover, we shed new light on the problem by proving an information-theoretic lower bound on the joint error of any domain adaptation method that attempts to learn invariant representations. Our result characterizes a fundamental tradeoff between learning invariant representations and achieving small joint error on both domains when the marginal label distributions differ from source to target. Finally, we conduct experiments on real-world datasets that corroborate our theoretical findings. We believe these insights are helpful in guiding the future design of domain adaptation and representation learning algorithms.