权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Auto-Tune: Structural Optimization of Machine Learning Frameworks for Large Datasets

Auto-Tune：大型数据集机器学习框架的结构优化

基本信息

批准号：
260351709
负责人：
Professor Dr.-Ing. Thomas Brox
金额：
--
依托单位：
Fachbereich IV: Informatik
依托单位国家：
德国
项目类别：
Priority Programmes
财政年份：
2014
资助国家：
德国
起止时间：
2013-12-31 至 2020-12-31
项目状态：
已结题

来源：
https://gepris.dfg.de/gepris/projekt/260351709?language=en
关键词：
Auto Tune Structural Optimization Machine

项目摘要

We aim to automate the design of machine learning algorithms, in order to facilitate their use by non-experts and in autonomous systems. Although automated adaptation is a core idea of machine learning, most algorithms still require a choice of external design parameters by an expert, which limits their commercial success. Our approach formalizes the search for good algorithm configurations as an optimization problem over the combined space of different machine learning algorithms and develops novel Bayesian optimization algorithms for its solution.This projects follows in the steps of our recent Auto-WEKA framework, which demonstrated that modern Bayesian optimization methods can provide non-experts with an automated (albeit computationally very expensive) method to identify state-of-the-art instantiations of complex learning frameworks. The next step is to make this approach feasible under realistic budget constraints, which, for modern-day (big) datasets and learning frameworks (especially deep learning) often imply that we cannot evaluate more than a few full model instantiations.We take inspiration from the way human practitioners attack a new learning problem: compare the dataset to those previously encountered, and evaluate some promising methods on subsets of the data, before then only constructing one or a few models on the full dataset.We plan to integrate all these components into a probabilistic model, using our recent Bayesian optimization algorithm of Entropy Search on a design space covering these dimensions to automatically derive a strategy that resembles the design strategy of a human expert. We will validate our approaches by improving upon the existing Auto-WEKA system, and by implementing a first approach for learning an effective deep network for a new dataset at the push of a button.We propose two theoretical research projects: 1) General Probabilistic Models of Algorithm PerformanceThis thread involves finding structured models that capture the highly structured interdependences across the often very high-dimensional parameter spaces of machine learning algorithms.2) Budget-Thrifty Hyperparameter OptimizationIt is often feasible to run machine learning algorithms in a cost-reduced form, either by thinning the dataset or by "switching off" certain parts of an algorithm. We aim to encode this possibility in a cost-aware optimization algorithm, which should then be able to automatically control the progression from rough prototyping to fine tuning.These two theoretical advances will enable two applied projects, whichare: 1) Automatic Machine Learning2) Automated structural optimization in computer vision, especially deep learning

我们的目标是自动化机器学习算法的设计，以方便非专家和自治系统的使用。虽然自动适应是机器学习的核心思想，但大多数算法仍然需要专家选择外部设计参数，这限制了它们的商业成功。我们的方法将寻找好的算法配置形式化为不同机器学习算法组合空间上的优化问题，并为其解决方案开发了新的贝叶斯优化算法。该项目遵循我们最近的Auto-WEKA框架的步骤，这表明，现代贝叶斯优化方法可以为非专家提供自动化的（尽管计算上非常昂贵）的方法来识别复杂学习框架的最先进的实例。下一步是在现实的预算限制下使这种方法可行，对于现代（大）数据集和学习框架，（尤其是深度学习）通常意味着我们只能评估几个完整的模型实例。我们从人类实践者解决新学习问题的方式中获得灵感：将数据集与以前遇到的数据集进行比较，并评估数据子集上的一些有前途的方法，在此之前，我们只在整个数据集上构建一个或几个模型。我们计划将所有这些组件集成到概率模型中，在覆盖这些维度的设计空间上使用我们最近的熵搜索贝叶斯优化算法来自动导出类似于人类专家的设计策略的策略。我们将通过改进现有的Auto-WEKA系统来验证我们的方法，并通过实现第一种方法来学习新数据集的有效深度网络。我们提出了两个理论研究项目：1）算法性能的一般概率模型这一线程涉及到寻找结构化模型，这些模型捕获了通常非常高的机器学习算法的多维参数空间。2）预算节约超参数优化通过精简数据集或通过“关闭”算法的某些部分，以成本降低的形式运行机器学习算法通常是可行的。我们的目标是将这种可能性编码到一个成本感知的优化算法中，然后该算法应该能够自动控制从粗略原型到微调的过程。这两个理论进步将实现两个应用项目，即：1）自动机器学习2）计算机视觉中的自动结构优化，特别是深度学习