Practical Adversarial Multivalid Conformal Prediction
Practical Adversarial Multivalid Conformal Prediction
复制标题
DOI：
10.48550/arxiv.2206.01067
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
O. Bastani;Varun Gupta;Christopher Jung;Georgy Noarov;Ramya Ramalingam;Aaron Roth
中科院分区：
文献类型：
--
作者：
O. Bastani;Varun Gupta;Christopher Jung;Georgy Noarov;Ramya Ramalingam;Aaron Roth
我们给出了一个简单的，通用的保形预测方法的顺序预测，实现目标的经验覆盖保证对adversarially选择的数据。它在计算上是轻量级的-与分裂的共形预测相当-但不需要有一个保持的验证集，因此所有数据都可以用于训练模型，从中导出共形分数。它在两个方面提供了比边际覆盖率更强的保证。首先，它给出了阈值校准的预测集，其具有正确的经验覆盖率，甚至以用于从适形分数形成预测集的阈值为条件。其次，用户可以指定特征空间的子集的任意集合-可能是相交的-并且覆盖保证也以这些子集中的每个子集的成员资格为条件。我们称我们的算法为MVP，即MultiValid Prediction的缩写。我们给出了理论和一套广泛的经验评价。
We give a simple, generic conformal prediction method for sequential prediction that achieves target empirical coverage guarantees against adversarially chosen data. It is computationally lightweight -- comparable to split conformal prediction -- but does not require having a held-out validation set, and so all data can be used for training models from which to derive a conformal score. It gives stronger than marginal coverage guarantees in two ways. First, it gives threshold calibrated prediction sets that have correct empirical coverage even conditional on the threshold used to form the prediction set from the conformal score. Second, the user can specify an arbitrary collection of subsets of the feature space -- possibly intersecting -- and the coverage guarantees also hold conditional on membership in each of these subsets. We call our algorithm MVP, short for MultiValid Prediction. We give both theory and an extensive set of empirical evaluations.