TrimTuner: Efficient Optimization of Machine Learning Jobs in the Cloud via Sub-Sampling
TrimTuner: Efficient Optimization of Machine Learning Jobs in the Cloud via Sub-Sampling
复制标题
DOI：
10.1109/mascots50786.2020.9285971
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Pedro Mendes;Maria Casimiro;P. Romano;D. Garlan
中科院分区：
文献类型：
--
作者：
Pedro Mendes;Maria Casimiro;P. Romano;D. Garlan
这项工作引入了 TrimTuner，这是第一个用于在云中优化机器学习作业的系统，利用子采样技术来降低优化过程的成本，同时考虑用户指定的约束。 TrimTuner 联合优化云和特定于应用程序的参数，与最先进的云优化工作不同，它避免了每次采样新配置时都使用完整训练集训练模型的需要。事实上，通过利用二次采样技术和比原始数据小 60 倍的数据集，我们表明 TrimTuner 可以将优化过程的成本降低多达 50 倍。此外，相对于使用子采样技术的超参数优化的最先进技术，TrimTuner 将推荐过程加快了 65 倍。这种改进的原因有两个：i）一种新颖的特定领域启发式方法，可以减少必须评估采集函数的配置数量； ii) 采用决策树集合，可以将推荐过程的速度提高一个数量级。
This work introduces TrimTuner, the first system for optimizing machine learning jobs in the cloud to exploit sub-sampling techniques to reduce the cost of the optimization process, while keeping into account user-specified constraints. TrimTuner jointly optimizes the cloud and application-specific parameters and, unlike state of the art works for cloud optimization, eschews the need to train the model with the full training set every time a new configuration is sampled. Indeed, by leveraging sub-sampling techniques and data-sets that are up to 60 x smaller than the original one, we show that TrimTuner can reduce the cost of the optimization process by up to 50 x. Further, TrimTuner speeds-up the recommendation process by 65 x with respect to state of the art techniques for hyperparameter optimization that use sub-sampling techniques. The reasons for this improvement are twofold: i) a novel domain specific heuristic that reduces the number of configurations for which the acquisition function has to be evaluated; ii) the adoption of an ensemble of decision trees that enables boosting the speed of the recommendation process by one additional order of magnitude.