High Dimensional Restrictive Federated Model Selection with multi-objective Bayesian Optimization over shifted distributions
High Dimensional Restrictive Federated Model Selection with multi-objective Bayesian Optimization over shifted distributions
复制标题
DOI：
10.1007/978-3-030-29516-5_48
复制
发表时间：
2019-02
期刊：
影响因子：
--
通讯作者：
Xudong Sun;Andrea Bommert;Florian Pfisterer;J. Rahnenführer;Michel Lang;B. Bischl
中科院分区：
文献类型：
--
作者：
Xudong Sun;Andrea Bommert;Florian Pfisterer;J. Rahnenführer;Michel Lang;B. Bischl
在这种情况下，提出了一种新的机器学习优化过程，称为限制性联邦模型选择（RFMS），例如，当来自医疗保健单位的数据不能离开其所在的站点，并且由于技术或隐私和信任问题而禁止在远程数据站点上执行训练算法时。为了在这种情况下进行临床研究，分析师可以只在本地数据站点上训练机器学习模型，但仍然可以以一定的成本执行统计查询，将机器学习模型发送到一些远程数据站点，并获得性能度量作为反馈，这可能是因为预测通常要便宜得多。与通过跨所有数据站点进行训练直接优化模型参数的联邦学习相比，RFMS仅在一个本地数据站点上训练模型参数，但在其他数据站点上联合优化超参数，因为超参数在机器学习性能中起着重要作用。目的是得到一个同时考虑本地和远程未知预测损失的Pareto最优模型，该模型可以很好地推广到不同的数据点。在这项工作中，我们特别考虑在数据站点上具有不同分布的高维数据。作为初步研究，利用贝叶斯优化特别是多目标贝叶斯优化指导自适应超参数优化过程来选择RFMS场景下的模型。经验结果表明，与利用本地和远程性能的方法相比，仅使用本地数据站点来调优超参数在数据站点之间的泛化效果较差。此外，就超大容量而言，多目标贝叶斯优化算法在其他候选数据站点中显示出跨多个数据站点的性能提高。
A novel machine learning optimization process coined Restrictive Federated Model Selection (RFMS) is proposed under the scenario, for example, when data from healthcare units can not leave the site it is situated on and it is forbidden to carry out training algorithms on remote data sites due to either technical or privacy and trust concerns. To carry out a clinical research in this scenario, an analyst could train a machine learning model only on local data site, but it is still possible to execute a statistical query at a certain cost in the form of sending a machine learning model to some of the remote data sites and get the performance measures as feedback, maybe due to prediction being usually much cheaper. Compared to federated learning, which is optimizing the model parameters directly by carrying out training across all data sites, RFMS trains model parameters only on one local data site but optimizes hyper parameters across other data sites jointly since hyper-parameters play an important role in machine learning performance. The aim is to get a Pareto optimal model with respective to both local and remote unseen prediction losses, which could generalize well across data sites. In this work, we specifically consider high dimensional data with different distributions over data sites. As an initial investigation, Bayesian Optimization especially multi-objective Bayesian Optimization is used to guide an adaptive hyper-parameter optimization process to select models under the RFMS scenario. Empirical results shows that solely using the local data site to tune hyper-parameters generalizes poorly across data sites, compared to methods that utilize the local and remote performances. Furthermore, in terms of hypervolumes, multi-objective Bayesian Optimization algorithms show increased performance across multiple data sites among other candidates.