Supercharging Imbalanced Data Learning With Energy-based Contrastive Representation Transfer
Supercharging Imbalanced Data Learning With Energy-based Contrastive Representation Transfer
复制标题
DOI：
--
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Zidi Xiu;Junya Chen;Ricardo Henao;B. Goldstein;L. Carin;Chenyang Tao
中科院分区：
文献类型：
--
作者：
Zidi Xiu;Junya Chen;Ricardo Henao;B. Goldstein;L. Carin;Chenyang Tao
处理严重的类不平衡对许多现实世界的应用程序提出了重大挑战，特别是当少数类的准确分类和泛化是主要兴趣时。在计算机视觉和NLP中，从具有长尾行为的数据集中学习是一个反复出现的主题，特别是对于自然发生的标签。现有的解决方案主要是通过抽样或加权调整来缓解极端的不平衡，或者施加归纳偏差来优先考虑可推广的关联。基于因果关系的不变性原则，我们从一个新的角度来提高样本效率和模型泛化。我们的贡献假设了一个元分布场景，其中标签条件特征的因果生成机制在不同标签之间是不变的。这种因果假设使知识从优势阶层有效地转移到代表性不足的阶层，即使他们的特征分布显示出明显的差异。这使我们能够利用因果数据增强过程来扩大少数族裔的代表性。我们的开发与现有的不平衡数据学习技术是正交的，因此可以无缝集成。针对最先进的解决方案，在广泛的合成和现实世界任务集上验证了所建议的方法。
Dealing with severe class imbalance poses a major challenge for many real-world applications, especially when the accurate classification and generalization of minority classes are of primary interest. In computer vision and NLP, learning from datasets with long-tail behavior is a recurring theme, especially for naturally occurring labels. Existing solutions mostly appeal to sampling or weighting adjustments to alleviate the extreme imbalance, or impose inductive bias to prioritize generalizable associations. Here we take a novel perspective to promote sample efficiency and model generalization based on the invariance principles of causality. Our contribution posits a meta-distributional scenario, where the causal generating mechanism for label-conditional features is invariant across different labels. Such causal assumption enables efficient knowledge transfer from the dominant classes to their under-represented counterparts, even if their feature distributions show apparent disparities. This allows us to leverage a causal data augmentation procedure to enlarge the representation of minority classes. Our development is orthogonal to the existing imbalanced data learning techniques thus can be seamlessly integrated. The proposed approach is validated on an extensive set of synthetic and real-world tasks against state-of-the-art solutions.