eFL-Boost: Efficient Federated Learning for Gradient Boosting Decision Trees
eFL-Boost: Efficient Federated Learning for Gradient Boosting Decision Trees
复制标题
DOI：
10.1109/access.2022.3169502
复制
发表时间：
2022
期刊：
影响因子：
3.9
通讯作者：
Fuki Yamamoto;S. Ozawa;Lihu Wang
中科院分区：
文献类型：
--
作者：
Fuki Yamamoto;S. Ozawa;Lihu Wang
隐私保护已经引起了越来越多的关注，隐私问题往往会阻碍灵活的数据利用。在大多数行业中，由于隐私问题，数据分布在多个组织中。联邦学习（FL）通过交流统计信息来实现跨组织机器学习，是用于解决这一问题的最先进技术。然而，对于FL中的梯度提升决策树（GBDT），在保持足够准确性的同时平衡通信效率和安全性仍然是一个未解决的问题。在本文中，我们提出了一个FL计划的GBDT，即，用于GBDT的高效FL（eFL-Boost），可最大限度地减少精度损失、通信成本和信息泄漏。该方案的重点是适当分配本地计算（由每个组织单独执行）和全局计算（由所有组织合作执行）时更新模型。已知树结构对于全局计算产生高通信成本，而叶权重不需要这样的成本，并且预期对准确性贡献相对更多。因此，在所提出的eFL-Boost中，在其中一个组织处局部地确定树结构，并且通过聚合所有组织的局部梯度来全局地计算叶权重。具体来说，eFL-Boost每次更新只需要三次通信，只有隐私风险低的统计信息才会泄露给其他组织。通过对公共数据集的性能评估（ROC AUC、Log loss和F1分数用作度量），所提出的eFL-Boost优于现有的方案，这些方案产生低通信成本，并且与不提供隐私保护的方案相当。
Privacy protection has attracted increasing attention, and privacy concerns often prevent flexible data utilization. In most industries, data are distributed across multiple organizations due to privacy concerns. Federated learning (FL), which enables cross-organizational machine learning by communicating statistical information, is a state-of-the-art technology that is used to solve this problem. However, for gradient boosting decision tree (GBDT) in FL, balancing communication efficiency and security while maintaining sufficient accuracy remains an unresolved problem. In this paper, we propose an FL scheme for GBDT, i.e., efficient FL for GBDT (eFL-Boost), which minimizes accuracy loss, communication costs, and information leakage. The proposed scheme focuses on appropriate allocation of local computation (performed individually by each organization) and global computation (performed cooperatively by all organizations) when updating a model. It is known that tree structures incur high communication costs for global computation, whereas leaf weights do not require such costs and are expected to contribute relatively more to accuracy. Thus, in the proposed eFL-Boost, a tree structure is determined locally at one of the organizations, and leaf weights are calculated globally by aggregating the local gradients of all organizations. Specifically, eFL-Boost requires only three communications per update, and only statistical information that has low privacy risk is leaked to other organizations. Through performance evaluation on public data sets (ROC AUC, Log loss, and F1-score are used as metrics), the proposed eFL-Boost outperforms existing schemes that incur low communication costs and was comparable to a scheme that offers no privacy protection.