Federated Boosted Decision Trees with Differential Privacy
Federated Boosted Decision Trees with Differential Privacy
复制标题
DOI：
10.1145/3548606.3560687
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
Samuel Maddock;Graham Cormode;Tianhao Wang;C. Maple;S. Jha
中科院分区：
文献类型：
--
作者：
Samuel Maddock;Graham Cormode;Tianhao Wang;C. Maple;S. Jha
人们对可扩展、安全和高效的隐私保护机器学习模型有着巨大的需求，这些模型可以在分布式数据上进行训练。虽然深度学习模型通常在集中式非安全环境中实现最佳结果，但当施加隐私和通信约束时，不同的模型可以表现出色。相反，基于树的方法（如XGBoost）因其高性能和易用性而备受关注;特别是，它们通常在表格数据上实现最先进的结果。因此，最近的几项工作都集中在通过同态加密（HE）和安全多方计算（MPC）等加密机制将XGBoost等梯度提升决策树（GBDT）模型转换为联邦设置。然而，这些并不总是提供正式的隐私保证，或者考虑全方位的超参数和实现设置。在这项工作中，我们实现了差分隐私（DP）下的GBDT模型。我们提出了一个通用的框架，捕捉和扩展现有的方法，不同的私人决策树。我们的框架的方法是量身定制的联邦设置，我们表明，通过精心选择的技术，它是可能实现非常高的实用性，同时保持强大的隐私水平。
There is great demand for scalable, secure, and efficient privacy-preserving machine learning models that can be trained over distributed data. While deep learning models typically achieve the best results in a centralized non-secure setting, different models can excel when privacy and communication constraints are imposed. Instead, tree-based approaches such as XGBoost have attracted much attention for their high performance and ease of use; in particular, they often achieve state-of-the-art results on tabular data. Consequently, several recent works have focused on translating Gradient Boosted Decision Tree (GBDT) models like XGBoost into federated settings, via cryptographic mechanisms such as Homomorphic Encryption (HE) and Secure Multi-Party Computation (MPC). However, these do not always provide formal privacy guarantees, or consider the full range of hyperparameters and implementation settings. In this work, we implement the GBDT model under Differential Privacy (DP). We propose a general framework that captures and extends existing approaches for differentially private decision trees. Our framework of methods is tailored to the federated setting, and we show that with a careful choice of techniques it is possible to achieve very high utility while maintaining strong levels of privacy.