Multi-Task Bayesian Optimization
Multi-Task Bayesian Optimization
复制标题
DOI：
--
复制
发表时间：
2013-12
期刊：
影响因子：
--
通讯作者：
Kevin Swersky;Jasper Snoek;Ryan P. Adams
中科院分区：
文献类型：
--
作者：
Kevin Swersky;Jasper Snoek;Ryan P. Adams
贝叶斯优化最近被提出作为自动调整机器学习模型的超参数的框架，并已被证明以令人印象深刻的简单性和效率产生最先进的性能。在本文中，我们探索是否有可能将从先前的优化中获得的知识转移到新的任务中，以便更有效地找到最优的超参数设置。我们的方法基于将多任务高斯过程扩展到贝叶斯优化框架。结果表明，与标准的单任务方法相比，该方法显著加快了优化过程。为了联合最小化跨多个任务的平均误差，我们进一步提出了算法的直接扩展，并演示了如何使用该扩展来极大地加速k-折交叉验证。最后，我们提出了一种新开发的获取功能，即熵搜索，以适应对成本敏感的多任务设置。我们通过利用一个小数据集来探索大型数据集的超参数设置，来演示这个新的获取函数的实用性。我们的算法动态地选择要查询的数据集，以便产生最多的单位成本信息。
Bayesian optimization has recently been proposed as a framework for automatically tuning the hyperparameters of machine learning models and has been shown to yield state-of-the-art performance with impressive ease and efficiency. In this paper, we explore whether it is possible to transfer the knowledge gained from previous optimizations to new tasks in order to find optimal hyperparameter settings more efficiently. Our approach is based on extending multi-task Gaussian processes to the framework of Bayesian optimization. We show that this method significantly speeds up the optimization process when compared to the standard single-task approach. We further propose a straightforward extension of our algorithm in order to jointly minimize the average error across multiple tasks and demonstrate how this can be used to greatly speed up k-fold cross-validation. Lastly, we propose an adaptation of a recently developed acquisition function, entropy search, to the cost-sensitive, multi-task setting. We demonstrate the utility of this new acquisition function by leveraging a small dataset to explore hyper-parameter settings for a large dataset. Our algorithm dynamically chooses which dataset to query in order to yield the most information per unit cost.