Stochastic Optimization for Spectral Risk Measures
Stochastic Optimization for Spectral Risk Measures
复制标题
DOI：
10.48550/arxiv.2212.05149
复制
发表时间：
2022-12
影响因子：
6.1
通讯作者：
Ronak R. Mehta;Vincent Roulet;Krishna Pillutla;Lang Liu;Zaïd Harchaoui
中科院分区：
文献类型：
--
作者：
Ronak R. Mehta;Vincent Roulet;Krishna Pillutla;Lang Liu;Zaïd Harchaoui
谱风险目标（也称为$L$-风险）允许学习系统在优化平均情况性能（如在经验风险最小化中）和任务的最坏情况性能之间进行插值。我们通过刻画它们的次微分并解决诸如次梯度估计的偏差和目标的非平滑性等挑战，开发了随机算法来优化这些量。我们从理论和实验上表明，诸如随机次梯度和对偶平均等现成的方法会受到偏差的阻碍，并且我们的方法优于它们。
Spectral risk objectives - also called $L$-risks - allow for learning systems to interpolate between optimizing average-case performance (as in empirical risk minimization) and worst-case performance on a task. We develop stochastic algorithms to optimize these quantities by characterizing their subdifferential and addressing challenges such as biasedness of subgradient estimates and non-smoothness of the objective. We show theoretically and experimentally that out-of-the-box approaches such as stochastic subgradient and dual averaging are hindered by bias and that our approach outperforms them.