AID: Active Distillation Machine to Leverage Pre-Trained Black-Box Models in Private Data Settings
AID: Active Distillation Machine to Leverage Pre-Trained Black-Box Models in Private Data Settings
复制标题
AID：主动蒸馏机在私人数据设置中利用预先训练的黑盒模型
DOI：
10.1145/3442381.3449944
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Sun, Jimeng
中科院分区：
文献类型：
--
作者：
Hoang, Trong Nghia;Hong, Shenda;Xiao, Cao;Low, Bryan;Sun, Jimeng
本文提出了一种主动蒸馏方法，供当地机构（例如医院）在其给定预算内找到最佳查询，从而将服务器上黑盒模型的预测知识蒸馏为具有透明参数化的本地代理。这使得当地机构能够在其本地环境中更好地理解黑盒模型的预测推理，或者利用其私有数据集（无法集中并输入到服务器模型中）进一步定制提取的知识。因此，所提出的方法解决了在具有严格专有限制的许多工业环境（例如医疗保健分析）中部署机器学习（ML）的几个挑战。其中包括：（1）服务器模型架构的不透明性，阻止本地用户在本地数据上下文中理解其预测推理； （2）将本地数据上传云端进行分析的成本和风险不断增加； (3)需要使用私有现场数据定制服务器模型。我们根据基准和现实世界的医疗数据评估了所提出的方法，其中观察到相对于现有的局部蒸馏方法有显着的改进。还提出了所提出方法的理论分析。
This paper presents an active distillation method for a local institution (e.g., hospital) to find the best queries within its given budget to distill an on-server black-box model’s predictive knowledge into a local surrogate with transparent parameterization. This allows local institutions to understand better the predictive reasoning of the black-box model in its own local context or to further customize the distilled knowledge with its private dataset that cannot be centralized and fed into the server model. The proposed method thus addresses several challenges of deploying machine learning (ML) in many industrial settings (e.g., healthcare analytics) with strong proprietary constraints. These include: (1) the opaqueness of the server model’s architecture which prevents local users from understanding its predictive reasoning in their local data contexts; (2) the increasing cost and risk of uploading local data on the cloud for analysis; and (3) the need to customize the server model with private onsite data. We evaluated the proposed method on both benchmark and real-world healthcare data where significant improvements over existing local distillation methods were observed. A theoretical analysis of the proposed method is also presented.
DOI：
--
发表时间：
2020-07
期刊：
--
影响因子：
--
作者：
T. Hoang;Chi Thanh Lam;B. Low;Patrick Jaillet
通讯作者：
T. Hoang;Chi Thanh Lam;B. Low;Patrick Jaillet