Predicting Out-of-Distribution Error with the Projection Norm
Predicting Out-of-Distribution Error with the Projection Norm
复制标题
DOI：
--
复制
发表时间：
2022-02
期刊：
影响因子：
--
通讯作者：
Yaodong Yu;Zitong Yang;Alexander Wei;Yi Ma;J. Steinhardt
中科院分区：
文献类型：
--
作者：
Yaodong Yu;Zitong Yang;Alexander Wei;Yi Ma;J. Steinhardt
我们提出了一个度量-投影范数-来预测模型在分布外（OOD）数据上的性能，而无需访问地面真实标签。Projection Norm首先使用模型预测来伪标记测试样本，然后在伪标记上训练新模型。新模型的参数与分布模型的差异越大，预测的OOD误差就越大。从经验上讲，我们的方法在图像和文本分类任务以及不同的网络架构上都优于现有方法。理论上，我们将我们的方法连接到过参数化线性模型的测试误差上的界。此外，我们发现投影范数是唯一一种在对抗性样本上实现非平凡检测性能的方法。我们的代码可在https://github.com/yaodongyu/ProjNorm上获得。
We propose a metric -- Projection Norm -- to predict a model's performance on out-of-distribution (OOD) data without access to ground truth labels. Projection Norm first uses model predictions to pseudo-label test samples and then trains a new model on the pseudo-labels. The more the new model's parameters differ from an in-distribution model, the greater the predicted OOD error. Empirically, our approach outperforms existing methods on both image and text classification tasks and across different network architectures. Theoretically, we connect our approach to a bound on the test error for overparameterized linear models. Furthermore, we find that Projection Norm is the only approach that achieves non-trivial detection performance on adversarial examples. Our code is available at https://github.com/yaodongyu/ProjNorm.