Interpreting Robust Optimization via Adversarial Influence Functions
Interpreting Robust Optimization via Adversarial Influence Functions
复制标题
DOI：
--
复制
发表时间：
2020-07
期刊：
影响因子：
--
通讯作者：
Zhun Deng;C. Dwork;Jialiang Wang;Linjun Zhang
中科院分区：
文献类型：
--
作者：
Zhun Deng;C. Dwork;Jialiang Wang;Linjun Zhang
鲁棒优化在当今数据科学中得到了广泛的应用，特别是在对抗性训练中。然而，与标准训练相比，很少有研究量化鲁棒优化如何改变优化器和预测损失。本文受稳健统计中影响函数的启发，引入对抗影响函数（AIF）作为研究稳健优化问题的工具。所提出的AIF具有封闭的形式，可以有效地计算。为了说明AIF的使用，我们将其应用于研究模型灵敏度-定义为在实施鲁棒优化后捕获自然数据上的预测损失的变化的量。我们使用AIF来分析模型复杂性和随机平滑如何影响特定模型的模型灵敏度。我们进一步推导出核回归的AIF，特别是应用于神经正切核，并通过实验证明了所提出的AIF的有效性。最后，将AIF理论推广到分布鲁棒优化问题。
Robust optimization has been widely used in nowadays data science, especially in adversarial training. However, little research has been done to quantify how robust optimization changes the optimizers and the prediction losses comparing to standard training. In this paper, inspired by the influence function in robust statistics, we introduce the Adversarial Influence Function (AIF) as a tool to investigate the solution produced by robust optimization. The proposed AIF enjoys a closed-form and can be calculated efficiently. To illustrate the usage of AIF, we apply it to study model sensitivity -- a quantity defined to capture the change of prediction losses on the natural data after implementing robust optimization. We use AIF to analyze how model complexity and randomized smoothing affect the model sensitivity with respect to specific models. We further derive AIF for kernel regressions, with a particular application to neural tangent kernels, and experimentally demonstrate the effectiveness of the proposed AIF. Lastly, the theories of AIF will be extended to distributional robust optimization.