Robust Optimal Transport with Applications in Generative Modeling and Domain Adaptation
Robust Optimal Transport with Applications in Generative Modeling and Domain Adaptation
复制标题
DOI：
--
复制
发表时间：
2020-10
期刊：
影响因子：
--
通讯作者：
Y. Balaji;Ramalingam Chellappa;S. Feizi
中科院分区：
文献类型：
--
作者：
Y. Balaji;Ramalingam Chellappa;S. Feizi
最佳运输（OT）距离（例如Wasserstein）已在gan和域适应等多个区域中使用。然而，OT对数据中的离群值（样本较大）非常敏感，因为在其目标函数中，每个样本（包括离群值）的称重相似，因此由于边缘约束而称重。为了解决这个问题，以前已经提出了具有不平衡边际约束的OT的强大配方。但是，由于其双重优化求解器的不稳定性，在深度学习问题（例如gan和域的适应性）等深度学习问题中采用这些方法具有挑战性。在本文中，我们通过得出适合现代深度学习应用程序的强大的OT优化的计算双重形式来解决这些问题。我们证明了我们在gan和域适应的两种应用中制定的有效性。我们的方法可以在因异常分布而损坏的嘈杂数据集上训练最先进的GAN模型。特别是，我们的优化计算训练样本的权重，反映了模型中这些样品的难度。在域的适应性中，与标准的对抗适应方法相比，我们的稳健OT配方可提高准确性。我们的代码可在此HTTPS URL上找到。
Optimal Transport (OT) distances such as Wasserstein have been used in several areas such as GANs and domain adaptation. OT, however, is very sensitive to outliers (samples with large noise) in the data since in its objective function, every sample, including outliers, is weighed similarly due to the marginal constraints. To remedy this issue, robust formulations of OT with unbalanced marginal constraints have previously been proposed. However, employing these methods in deep learning problems such as GANs and domain adaptation is challenging due to the instability of their dual optimization solvers. In this paper, we resolve these issues by deriving a computationally-efficient dual form of the robust OT optimization that is amenable to modern deep learning applications. We demonstrate the effectiveness of our formulation in two applications of GANs and domain adaptation. Our approach can train state-of-the-art GAN models on noisy datasets corrupted with outlier distributions. In particular, our optimization computes weights for training samples reflecting how difficult it is for those samples to be generated in the model. In domain adaptation, our robust OT formulation leads to improved accuracy compared to the standard adversarial adaptation methods. Our code is available at this https URL.