Batch Multivalid Conformal Prediction
Batch Multivalid Conformal Prediction
复制标题
DOI：
10.48550/arxiv.2209.15145
复制
发表时间：
2022-09
期刊：
影响因子：
--
通讯作者：
Christopher Jung;Georgy Noarov;Ramya Ramalingam;Aaron Roth
中科院分区：
文献类型：
--
作者：
Christopher Jung;Georgy Noarov;Ramya Ramalingam;Aaron Roth
我们开发了快速的无分布共形预测算法，用于在批处理设置中获得可交换数据的多瓦化覆盖范围。通过两种方式，多估计保证的保证比边缘覆盖范围更强：（1）他们以团体成员身份持有有条件的条件 - 也就是说，目标覆盖率$ 1- \ alpha $在每个任意（潜在的相交）中有条件地持有有条件的成员资格在特征空间中区域的有限集合中的$ \ mathcal {g} $组中的组。 （2）他们以用于在给定示例上产生预测的阈值的阈值的值均匀。实际上，即使同时根据小组成员资格和门槛值进行条件，多估计覆盖范围也可以保证。我们给出了两种算法：两者都将输入作为任意的不符合分数，并任意收集可能相交的组$ \ Mathcal {g} $，然后可以配备任意的黑盒预测指标，并配备预测集。我们的第一个算法（batchGCP）是分位数回归的直接扩展，需要仅解决单个凸的最小化问题，并产生一个估计器，该估计器在$ \ Mathcal {g} $中对每个组都有群体条件保证。我们的第二个算法（batchMVP）是迭代的，并提供了多千差联综合预测的全部保证：预测集在小组成员资格和非合规性阈值中有条件有效。我们在广泛的实验集中评估了两种算法的性能。代码复制我们所有实验
We develop fast distribution-free conformal prediction algorithms for obtaining multivalid coverage on exchangeable data in the batch setting. Multivalid coverage guarantees are stronger than marginal coverage guarantees in two ways: (1) They hold even conditional on group membership -- that is, the target coverage level $1-\alpha$ holds conditionally on membership in each of an arbitrary (potentially intersecting) group in a finite collection $\mathcal{G}$ of regions in the feature space. (2) They hold even conditional on the value of the threshold used to produce the prediction set on a given example. In fact multivalid coverage guarantees hold even when conditioning on group membership and threshold value simultaneously. We give two algorithms: both take as input an arbitrary non-conformity score and an arbitrary collection of possibly intersecting groups $\mathcal{G}$, and then can equip arbitrary black-box predictors with prediction sets. Our first algorithm (BatchGCP) is a direct extension of quantile regression, needs to solve only a single convex minimization problem, and produces an estimator which has group-conditional guarantees for each group in $\mathcal{G}$. Our second algorithm (BatchMVP) is iterative, and gives the full guarantees of multivalid conformal prediction: prediction sets that are valid conditionally both on group membership and non-conformity threshold. We evaluate the performance of both of our algorithms in an extensive set of experiments. Code to replicate all of our experiments can be found at https://github.com/ProgBelarus/BatchMultivalidConformal