Invariance Learning based on Label Hierarchy-猫眼课题宝

DOI：

10.48550/arxiv.2203.15549

发表时间：

2022-03

期刊：

ArXiv

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

S. Toyota;K. Fukumizu

深度神经网络继承了嵌入在训练数据中的虚假相关性，因此可能无法在看不见的域（或环境）上预测所需的标签，这些域与训练中使用的域具有不同的分布。不变性学习（IL）是近年来为了克服这一缺点而发展起来的。使用许多领域的训练数据，IL估计出这样一个预测器，它对领域的变化是不变的。然而，对多领域训练数据的要求是IL的一个很强的限制，因为它通常需要很高的标注成本。我们提出了一个新的IL框架来克服这个问题。假设一个更高层次的分类任务有来自多个域的数据可用，且标记成本较低，我们用单个域的训练数据估计目标分类任务的不变预测器。此外，我们提出了两种选择不变性正则化超参数的交叉验证方法，以解决现有的正则化方法中没有很好处理的超参数选择问题。经验证明了该框架的有效性，包括交叉验证，并在一定条件下证明了超参数选择的正确性。

Deep Neural Networks inherit spurious correlations embedded in training data and hence may fail to predict desired labels on unseen domains (or environments), which have different distributions from the domain used in training. Invariance Learning (IL) has been developed recently to overcome this shortcoming; using training data in many domains, IL estimates such a predictor that is invariant to a change of domain. However, the requirement of training data in multiple domains is a strong restriction of IL, since it often needs high annotation cost. We propose a novel IL framework to overcome this problem. Assuming the availability of data from multiple domains for a higher level of classification task, for which the labeling cost is low, we estimate an invariant predictor for the target classification task with training data in a single domain. Additionally, we propose two cross-validation methods for selecting hyperparameters of invariance regularization to solve the issue of hyperparameter selection, which has not been handled properly in existing IL methods. The effectiveness of the proposed framework, including the cross-validation, is demonstrated empirically, and the correctness of the hyperparameter selection is proved under some conditions.

Invariance Learning based on Label Hierarchy

文献摘要

相似文献