OccamNets: Mitigating Dataset Bias by Favoring Simpler Hypotheses
OccamNets: Mitigating Dataset Bias by Favoring Simpler Hypotheses
复制标题
DOI：
10.48550/arxiv.2204.02426
复制
发表时间：
2022-04
期刊：
影响因子：
1.9
通讯作者：
Robik Shrestha;Kushal Kafle;Christopher Kanan
中科院分区：
文献类型：
--
作者：
Robik Shrestha;Kushal Kafle;Christopher Kanan
数据集偏差和虚假相关性会严重损害深度神经网络的泛化能力。许多先前的努力已经使用替代损失函数或专注于罕见模式的采样策略来解决这个问题。我们提出了一个新的方向：修改网络架构以施加归纳偏差，使网络对数据集偏差具有鲁棒性。具体来说，我们提出了 OccamNet，它在设计上偏向于支持更简单的解决方案。 OccamNets 有两个归纳偏差。首先，他们倾向于根据单个示例的需要使用尽可能少的网络深度。其次，他们偏向于使用较少的图像位置进行预测。虽然 OccamNet 偏向于更简单的假设，但如有必要，它们可以学习更复杂的假设。在实验中，OccamNets 优于或竞争在不包含这些归纳偏差的架构上运行的最先进方法。此外，我们证明，当最先进的去偏方法与 OccamNets 相结合时，结果会进一步改善。
Dataset bias and spurious correlations can significantly impair generalization in deep neural networks. Many prior efforts have addressed this problem using either alternative loss functions or sampling strategies that focus on rare patterns. We propose a new direction: modifying the network architecture to impose inductive biases that make the network robust to dataset bias. Specifically, we propose OccamNets, which are biased to favor simpler solutions by design. OccamNets have two inductive biases. First, they are biased to use as little network depth as needed for an individual example. Second, they are biased toward using fewer image locations for prediction. While OccamNets are biased toward simpler hypotheses, they can learn more complex hypotheses if necessary. In experiments, OccamNets outperform or rival state-of-the-art methods run on architectures that do not incorporate these inductive biases. Furthermore, we demonstrate that when the state-of-the-art debiasing methods are combined with OccamNets results further improve.