Boosted CVaR Classification
Boosted CVaR Classification
复制标题
DOI：
--
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
Runtian Zhai;Chen Dan;A. Suggala;Zico Kolter;Pradeep Ravikumar
中科院分区：
文献类型：
--
作者：
Runtian Zhai;Chen Dan;A. Suggala;Zico Kolter;Pradeep Ravikumar
许多现代机器学习任务需要具有高尾部性能的模型，即在数据集中表现最差的样本上也能有出色表现。这一问题在算法公平性、类别不平衡以及风险敏感决策等领域已得到广泛研究。一种提升模型尾部性能的常用方法是最小化条件风险价值（CVaR，Conditional Value at Risk）损失，该损失计算的是损失尾部的平均风险。然而，对于通过零一损失来评估模型的分类任务，我们证明了如果分类器是确定性的，那么平均零一损失的最小化器也会使CVaR零一损失最小化，这表明在没有额外假设的情况下，最小化CVaR损失并无益处。我们通过在随机化分类器上最小化CVaR损失来规避这一不利结果，因为对于随机化分类器而言，平均零一损失的最小化器与CVaR零一损失的最小化器不再相同，所以最小化后者能够带来更好的尾部性能。为了学习这类随机化分类器，我们提出了“提升式CVaR分类”框架，它源于CVaR与一种名为LPBoost的经典提升算法之间的直接联系。基于此框架，我们设计了一种名为$\alpha$-AdaLPBoost的算法。我们在四个基准数据集上对所提出的算法进行了实证评估，结果表明，与确定性模型训练方法相比，该算法实现了更高的尾部性能。
Many modern machine learning tasks require models with high tail performance, i.e. high performance over the worst-off samples in the dataset. This problem has been widely studied in fields such as algorithmic fairness, class imbalance, and risk-sensitive decision making. A popular approach to maximize the model's tail performance is to minimize the CVaR (Conditional Value at Risk) loss, which computes the average risk over the tails of the loss. However, for classification tasks where models are evaluated by the zero-one loss, we show that if the classifiers are deterministic, then the minimizer of the average zero-one loss also minimizes the CVaR zero-one loss, suggesting that CVaR loss minimization is not helpful without additional assumptions. We circumvent this negative result by minimizing the CVaR loss over randomized classifiers, for which the minimizers of the average zero-one loss and the CVaR zero-one loss are no longer the same, so minimizing the latter can lead to better tail performance. To learn such randomized classifiers, we propose the Boosted CVaR Classification framework which is motivated by a direct relationship between CVaR and a classical boosting algorithm called LPBoost. Based on this framework, we design an algorithm called $\alpha$-AdaLPBoost. We empirically evaluate our proposed algorithm on four benchmark datasets and show that it achieves higher tail performance than deterministic model training methods.