Descent-to-Delete: Gradient-Based Methods for Machine Unlearning
Descent-to-Delete: Gradient-Based Methods for Machine Unlearning
复制标题
DOI：
--
复制
发表时间：
2020-07
期刊：
影响因子：
--
通讯作者：
Seth Neel;Aaron Roth;Saeed Sharifi-Malvajerdi
中科院分区：
文献类型：
--
作者：
Seth Neel;Aaron Roth;Saeed Sharifi-Malvajerdi
研究了凸模型的数据删除问题。通过利用凸优化和水库采样的技术，我们给出了第一个数据删除算法，该算法能够处理任意长的对抗性更新序列，同时保证每次删除运行时间和稳态错误不会随着更新序列的长度而增长。我们还引入了几个新的概念区别：例如，我们可以要求删除后，优化算法保持的整个状态在统计上与重新训练后的状态无法区分，或者我们可以要求更弱的条件，即只有可观察输出在统计上与重新训练后的可观察输出无法区分。我们能够给出更有效的删除算法在此较弱的删除标准。
We study the data deletion problem for convex models. By leveraging techniques from convex optimization and reservoir sampling, we give the first data deletion algorithms that are able to handle an arbitrarily long sequence of adversarial updates while promising both per-deletion run-time and steady-state error that do not grow with the length of the update sequence. We also introduce several new conceptual distinctions: for example, we can ask that after a deletion, the entire state maintained by the optimization algorithm is statistically indistinguishable from the state that would have resulted had we retrained, or we can ask for the weaker condition that only the observable output is statistically indistinguishable from the observable output that would have resulted from retraining. We are able to give more efficient deletion algorithms under this weaker deletion criterion.