End-to-End Deep Learning of Optimization Heuristics
End-to-End Deep Learning of Optimization Heuristics
复制标题
DOI：
10.1109/pact.2017.24
复制
发表时间：
2017-09
期刊：
影响因子：
--
通讯作者：
Chris Cummins;Pavlos Petoumenos;Zheng Wang;Hugh Leather
中科院分区：
文献类型：
--
作者：
Chris Cummins;Pavlos Petoumenos;Zheng Wang;Hugh Leather
精确的自动优化算法对于处理现代硬件和软件的复杂性和多样性是必要的。机器学习是一种经过验证的学习此类知识的技术，但它的成功取决于所用特征的质量。这些特性必须由开发人员通过结合专业领域知识和反复试验来手工制作。这使得最终模型的质量直接依赖于系统架构师的技能和可用时间。我们开发了一个深度神经网络，它可以在原始代码上学习算法，完全不使用代码特征。神经网络同时构造代码的适当表示，并学习如何最好地优化，消除了手动创建功能的需要。此外，我们还展示了我们的神经网络可以将学习从一个优化问题转移到另一个优化问题，提高新模型的准确性，而无需人类专家的帮助。我们将自动生成的算法与专家手工挑选的特征进行了比较。我们研究两个具有挑战性的任务：预测异构并行和GPU线程粗化因素的最佳映射。在89%的情况下，我们完全自动化的质量匹配或超过了使用手工制作的功能的最先进的预测模型，平均提高了14%和12%的性能，而无需在设计功能上花费人力。
Accurate automatic optimization heuristics are necessary for dealing with thecomplexity and diversity of modern hardware and software. Machine learning is aproven technique for learning such heuristics, but its success is bound by thequality of the features used. These features must be hand crafted by developersthrough a combination of expert domain knowledge and trial and error. This makesthe quality of the final model directly dependent on the skill and availabletime of the system architect.Our work introduces a better way for building heuristics. We develop a deepneural network that learns heuristics over raw code, entirely without using codefeatures. The neural network simultaneously constructs appropriaterepresentations of the code and learns how best to optimize, removing the needfor manual feature creation. Further, we show that our neural nets can transferlearning from one optimization problem to another, improving the accuracy of newmodels, without the help of human experts.We compare the effectiveness of our automatically generated heuristics againstones with features hand-picked by experts. We examine two challenging tasks:predicting optimal mapping for heterogeneous parallelism and GPU threadcoarsening factors. In 89% of the cases, the quality of our fully automaticheuristics matches or surpasses that of state-of-the-art predictive models usinghand-crafted features, providing on average 14% and 12% more performance withno human effort expended on designing features.