Democratizing Data Science through Interactive Curation of ML Pipelines
Democratizing Data Science through Interactive Curation of ML Pipelines
复制标题
DOI：
10.1145/3299869.3319863
复制
发表时间：
2019-06
期刊：
影响因子：
--
通讯作者：
Zeyuan Shang;Emanuel Zgraggen;Benedetto Buratti;Ferdinand Kossmann;P. Eichmann;Yeounoh Chung;Carsten Binnig;E. Upfal;Tim Kraska
中科院分区：
文献类型：
--
作者：
Zeyuan Shang;Emanuel Zgraggen;Benedetto Buratti;Ferdinand Kossmann;P. Eichmann;Yeounoh Chung;Carsten Binnig;E. Upfal;Tim Kraska
统计知识和领域专业知识是从数据中提取可操作的见解的关键，但这些技能很少共存。在机器学习中，只有通过认真的数据预处理、超参数调整和模型选择才能获得高质量的结果。领域专家经常被这种复杂性淹没，事实上阻碍了ML技术在其他领域的更广泛采用。现有的图书馆声称可以解决这个问题，但仍然需要训练有素的从业人员。这些框架涉及繁重的数据准备步骤，而且往往太慢，无法得到用户的交互反馈，严重限制了此类系统的范围。在本文中，我们介绍了第一个交互式自动机器学习工具AlMountain Meadow。我们的系统的独特之处不仅在于对交互性的关注，还在于系统和算法相结合的设计方法；我们一方面利用查询优化的思想，另一方面结合基于成本的多臂Bandits和贝叶斯优化设计了新颖的选择和剪枝策略。我们在300多个数据集上对我们的系统进行了评估，并与其他AutoML工具(包括当前的NIPS Winner)以及专家解决方案进行了比较。AlMountain Meadow不仅能够显著超越其他AutoML系统，同时-与其他系统形成对比-提供交互延迟，而且在80%的情况下，专家解决方案的表现优于我们以前从未见过的数据集。
Statistical knowledge and domain expertise are key to extract actionable insights out of data, yet such skills rarely coexist together. In Machine Learning, high-quality results are only attainable via mindful data preprocessing, hyperparameter tuning and model selection. Domain experts are often overwhelmed by such complexity, de-facto inhibiting a wider adoption of ML techniques in other fields. Existing libraries that claim to solve this problem, still require well-trained practitioners. Those frameworks involve heavy data preparation steps and are often too slow for interactive feedback from the user, severely limiting the scope of such systems. In this paper we present Alpine Meadow, a first Interactive Automated Machine Learning tool. What makes our system unique is not only the focus on interactivity, but also the combined systemic and algorithmic design approach; on one hand we leverage ideas from query optimization, on the other we devise novel selection and pruning strategies combining cost-based Multi-Armed Bandits and Bayesian Optimization. We evaluate our system on over 300 datasets and compare against other AutoML tools, including the current NIPS winner, as well as expert solutions. Not only is Alpine Meadow able to significantly outperform the other AutoML systems while --- in contrast to the other systems --- providing interactive latencies, but also outperforms in 80% of the cases expert solutions over data sets we have never seen before.