Revisiting Distributionally Robust Supervised Learning in Classification
Revisiting Distributionally Robust Supervised Learning in Classification
复制标题
重新审视分类中的分布式鲁棒监督学习
DOI：
--
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
Masashi Sugiyama
中科院分区：
文献类型：
--
作者：
Weihua Hu;Gang Niu;Issei Sato;Masashi Sugiyama
分布式稳健监督学习(DRSL)是建立可靠的机器学习系统所必需的。当机器学习被部署在现实世界中时，其性能可能会显著下降，因为测试数据可能遵循与训练数据不同的分布。以前的DRSL最大限度地减少了最坏情况下测试分布的损失。然而，我们的理论分析表明，在分类场景中，以前的DRSL本质上归结为普通的经验风险最小化。这意味着，即使先前的DRSL被设计成对来自训练数据集的分布漂移具有健壮性，它最终也会精确地针对给定的训练数据来学习分类器。为了学习实用的稳健分类器，我们的理论分析激励我们从结构上约束DRSL所考虑的分布偏移。为此，我们提出了一种新的DRSL，它可以结合关于分布平移的结构性假设，并且可以基于这些假设学习有用的稳健决策边界。我们推导了有效的基于梯度的优化算法，并建立了模型参数的收敛速度以及估计误差的阶数。通过实验验证了我们的DRSL的有效性。
Distributionally Robust Supervised Learning (DRSL) is necessary for building reliable machine learning systems. When machine learning is deployed in the real world, its performance can be significantly degraded because test data may follow a different distribution from training data. Previous DRSL minimizes the loss for the worst-case test distribution. However, our theoretical analyses show that the previous DRSL essentially reduces to ordinary empirical risk minimization in a classification scenario. This implies that the previous DRSL ends up learning classifiers exactly for the given training data even though it is designed to be robust to distribution shift from the training dataset. In order to learn practically useful robust classifiers, our theoretical analyses motivate us to structurally constrain the distribution shift considered by DRSL. To this end, we propose novel DRSL which can incorporate the structural assumptions on distribution shift and that can learn useful robust decision boundaries based on the assumptions. We derive efficient gradient-based optimization algorithms and establish the convergence rate of the model parameter as well as the order of the estimation error for our DRSL. The effectiveness of our DRSL is demonstrated through experiments.
DOI：
10.1109/cvpr.2015.7298619
发表时间：
2015-06
期刊：
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
影响因子：
--
作者：
M. Ristin;Juergen Gall;M. Guillaumin;L. Gool
通讯作者：
M. Ristin;Juergen Gall;M. Guillaumin;L. Gool