Machine Unlearning in Gradient Boosting Decision Trees
Machine Unlearning in Gradient Boosting Decision Trees
复制标题
DOI：
10.1145/3580305.3599420
复制
发表时间：
2023-08
期刊：
影响因子：
--
通讯作者：
Huawei Lin;Jun Woo Chung;Yingjie Lao;Weijie Zhao
中科院分区：
文献类型：
--
作者：
Huawei Lin;Jun Woo Chung;Yingjie Lao;Weijie Zhao
各种机器学习应用程序将用户的数据训练模型。最近执行的立法要求公司根据请求删除用户的数据，即被遗忘的权利。在机器学习的背景下，受过训练的模型可能会记住培训数据。机器学习算法必须能够学习要求删除要求的用户数据以满足要求。梯度提升决策树（GBDT）是许多机器学习应用程序中广泛部署的模型。但是，很少有研究研究GBDT上的学习。本文提出了一个新颖的GBDT学习框架。据我们所知，这是考虑机器在GBDT上学习的第一项工作。将DNN的未学习方法传递到GBDT设置并不直接。我们正式化了机器学习问题及其轻松版本。我们提出了一个未经学习的框架，该框架可以有效地有效地学习给定的数据集合，而无需从头开始重新录制模型。我们将一系列技术（包括随机的拆分点选择和随机分配层训练）介绍给原始树型模型的训练过程，以确保训练有素的模型需要在学习期间很少的子树重新培训。我们研究了培训期间的中间数据和统计数据，以存储为辅助数据结构，以便我们可以立即确定是否需要在不触摸原始培训数据集的情况下重新训练子树。此外，提出了一种懒惰的更新技术，作为未学习时间和模型功能之间的权衡。我们在公共数据集上实验评估我们提出的方法。经验结果证实了我们框架的有效性。
Various machine learning applications take users' data to train the models. Recently enforced legislation requires companies to remove users' data upon requests, i.e.,the right to be forgotten. In the context of machine learning, the trained model potentially memorizes the training data. Machine learning algorithms have to be able to unlearn the user data that are requested to delete to meet the requirement. Gradient Boosting Decision Trees (GBDT) is a widely deployed model in many machine learning applications. However, few studies investigate the unlearning on GBDT. This paper proposes a novel unlearning framework for GBDT. To the best of our knowledge, this is the first work that considers machine unlearning on GBDT. It is not straightforward to transfer the unlearning methods of DNN to GBDT settings. We formalized the machine unlearning problem and its relaxed version. We propose an unlearning framework that efficiently and effectively unlearns a given collection of data without retraining the model from scratch. We introduce a collection of techniques, including random split point selection and random partitioning layers training, to the training process of the original tree models to ensure that the trained model requires few subtree retrainings during the unlearning. We investigate the intermediate data and statistics to store as an auxiliary data structure during the training so that we can immediately determine if a subtree is required to be retrained without touching the original training dataset. Furthermore, a lazy update technique is proposed as a trade-off between unlearning time and model functionality. We experimentally evaluate our proposed methods on public datasets. The empirical results confirm the effectiveness of our framework.