When AUC meets DRO: Optimizing Partial AUC for Deep Learning with Non-Convex Convergence Guarantee
When AUC meets DRO: Optimizing Partial AUC for Deep Learning with Non-Convex Convergence Guarantee
复制标题
DOI：
10.48550/arxiv.2203.00176
复制
发表时间：
2022-03
期刊：
影响因子：
--
通讯作者：
Dixian Zhu;Gang Li;Bokun Wang;Xiaodong Wu;Tianbao Yang
中科院分区：
文献类型：
--
作者：
Dixian Zhu;Gang Li;Bokun Wang;Xiaodong Wu;Tianbao Yang
本文提出了一种适用于深度学习的系统、高效的基于梯度的单向和双向部分AUC最大化方法。我们提出了新的pAUC替代目标的公式，通过使用分布稳健优化(DRO)来定义每个独立正数据的损失。我们考虑了两种DRO公式，一种是基于条件风险值(CVAR)的，它给出了pAUC的一个非光滑但精确的估计量；另一种是基于KL散度正则化的DRO，它给出了pAUC的一个不精确但光滑(软)的估计量。对于单向和双向pAUC最大化问题，我们分别提出了两种算法，并证明了它们在优化这两个公式上的收敛。实验证明了本文提出的pAUC最大化算法在不同数据集上进行深度学习的有效性。
In this paper, we propose systematic and efficient gradient-based methods for both one-way and two-way partial AUC (pAUC) maximization that are applicable to deep learning. We propose new formulations of pAUC surrogate objectives by using the distributionally robust optimization (DRO) to define the loss for each individual positive data. We consider two formulations of DRO, one of which is based on conditional-value-at-risk (CVaR) that yields a non-smooth but exact estimator for pAUC, and another one is based on a KL divergence regularized DRO that yields an inexact but smooth (soft) estimator for pAUC. For both one-way and two-way pAUC maximization, we propose two algorithms and prove their convergence for optimizing their two formulations, respectively. Experiments demonstrate the effectiveness of the proposed algorithms for pAUC maximization for deep learning on various datasets.