ROBUST WASSERSTEIN PROFILE INFERENCE AND APPLICATIONS TO MACHINE LEARNING
ROBUST WASSERSTEIN PROFILE INFERENCE AND APPLICATIONS TO MACHINE LEARNING
复制标题
DOI：
10.1017/jpr.2019.49
复制
发表时间：
2019-09-01
影响因子：
1
通讯作者：
Murthy, Karthyek
中科院分区：
文献类型：
--
作者：
Blanchet, Jose;Kang, Yang;Murthy, Karthyek
我们证明了几个机器学习估计器，包括平方根最小绝对收缩和选择以及正则化逻辑回归，可以表示为分布鲁棒优化问题的解。相关的不确定性区域基于适当定义的沃瑟斯坦距离。因此，我们的表征允许我们将正则化视为引入人工对手的结果，该对手干扰经验分布以解释损失估计中的样本外效应。此外，我们引入了RWPI（鲁棒Wasserstein剖面推理），这是一种新的推理方法，它将经验似然启发的方法的使用扩展到最优运输成本的设置（其中Wasserstein距离是一个特殊情况）。我们使用RWPI来展示如何最佳地选择不确定区域的大小，因此我们能够在不使用交叉验证的情况下为这些机器学习估计器选择正则化参数。数值实验也验证了我们的理论结论。
We show that several machine learning estimators, including square-root least absolute shrinkage and selection and regularized logistic regression, can be represented as solutions to distributionally robust optimization problems. The associated uncertainty regions are based on suitably defined Wasserstein distances. Hence, our representations allow us to view regularization as a result of introducing an artificial adversary that perturbs the empirical distribution to account for out-of-sample effects in loss estimation. In addition, we introduce RWPI (robust Wasserstein profile inference), a novel inference methodology which extends the use of methods inspired by empirical likelihood to the setting of optimal transport costs (of which Wasserstein distances are a particular case). We use RWPI to show how to optimally select the size of uncertainty regions, and as a consequence we are able to choose regularization parameters for these machine learning estimators without the use of cross validation. Numerical experiments are also given to validate our theoretical findings.