Beyond Manual Tuning of Hyperparameters
Beyond Manual Tuning of Hyperparameters
复制标题
DOI：
10.1007/s13218-015-0381-0
复制
发表时间：
2015-11-01
影响因子：
2.9
通讯作者：
Schmidt-Thieme, Lars
中科院分区：
文献类型：
--
作者：
Hutter, Frank;Luecke, Joerg;Schmidt-Thieme, Lars
手工制作的机器学习系统在许多应用中的成功提出了使机器学习算法更加自主的问题，即将专家输入的要求降至最低。我们讨论了实现这一目标的两种策略：（1）超参数的自动优化（包括特征选择、预处理、模型选择等机制）和（2）开发具有减少的超参数集的算法。由于许多研究方向（例如深度学习）显示出算法越来越复杂、超参数越来越多的趋势，因此对这两种策略的需求不断增加。我们回顾了最近的超参数优化方法，并讨论了数据驱动的方法，以避免使用无监督学习引入超参数。最后我们讨论了这些互补策略如何协同工作，代表了一种非常有前途的自主机器学习方法。
The success of hand-crafted machine learning systems in many applications raises the question of making machine learning algorithms more autonomous, i.e., to reduce the requirement of expert input to a minimum. We discuss two strategies towards this goal: (1) automated optimization of hyperparameters (including mechanisms for feature selection, preprocessing, model selection, etc) and (2) the development of algorithms with reduced sets of hyperparameters. Since many research directions (e.g., deep learning), show a tendency towards increasingly complex algorithms with more and more hyperparamters, the demand for both of these strategies continuously increases. We review recent hyperparameter optimization methods and discuss data-driven approaches to avoid the introduction of hyperparameters using unsupervised learning. We end in discussing how these complementary strategies can work hand-in-hand, representing a very promising approach towards autonomous machine learning.