Avoiding spurious correlations via logit correction-猫眼课题宝

DOI：

10.48550/arxiv.2212.01433

发表时间：

2022-12

期刊：

ArXiv

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Sheng Liu;Xu Zhang-;Nitesh Sekhar;Yue Wu;Prateek Singhal;C. Fernandez‐Granda

经验研究表明，使用经验风险最小化（ERM）训练的机器学习模型通常依赖于可能与类标签虚假相关的属性。这样的模型通常导致在对缺乏这种相关性的数据进行推断期间性能较差。在这项工作中，我们明确考虑了一种情况，即大多数训练数据中存在潜在的虚假相关性。与现有的方法相比，这些方法使用ERM模型输出来检测没有虚假相关性的样本，并对这些样本进行逐层加权或上采样，我们提出了logit校正（LC）损失，这是对softmax交叉熵损失的一种简单而有效的改进，以校正样本logit。我们证明了最小化LC损失相当于最大化组平衡精度，因此所提出的LC可以减轻虚假相关的负面影响。我们广泛的实验结果进一步表明，所提出的LC损失优于国家的最先进的解决方案，在多个流行的基准的大幅度，平均5.5%的绝对改善，没有访问虚假的属性标签。LC也与使用属性标签的Oracle方法竞争。代码可在https://github.com/shengliu66/LC上获得。

Empirical studies suggest that machine learning models trained with empirical risk minimization (ERM) often rely on attributes that may be spuriously correlated with the class labels. Such models typically lead to poor performance during inference for data lacking such correlations. In this work, we explicitly consider a situation where potential spurious correlations are present in the majority of training data. In contrast with existing approaches, which use the ERM model outputs to detect the samples without spurious correlations and either heuristically upweight or upsample those samples, we propose the logit correction (LC) loss, a simple yet effective improvement on the softmax cross-entropy loss, to correct the sample logit. We demonstrate that minimizing the LC loss is equivalent to maximizing the group-balanced accuracy, so the proposed LC could mitigate the negative impacts of spurious correlations. Our extensive experimental results further reveal that the proposed LC loss outperforms state-of-the-art solutions on multiple popular benchmarks by a large margin, an average 5.5\% absolute improvement, without access to spurious attribute labels. LC is also competitive with oracle methods that make use of the attribute labels. Code is available at https://github.com/shengliu66/LC.

Avoiding spurious correlations via logit correction

文献摘要

相似文献