Improving efficiency and sustainability of automated machine learning

点击翻译按钮获取中文摘要

In the recent years, automated machine learning (AutoML) has been proven successful in discovering efficient neural networks, improving upon manually designed networks in a number of machine learning tasks - such as image classification, super-resolution, automatic speech recognition, and more. However, early AutoML techniques were extremely demanding in terms of computational resources, often necessitating orders of magnitude more GPU-hours than a normal training. Although significant improvements have been made in making AutoML more efficient since then, the time and resources required to automatically optimise a neural network's structure still remain one of the main factors limiting applicability of AutoML, especially when very large search spaces and datasets are considered. This project aims to investigate the scalability and related limitations of the existing approaches with a special focus on those that utilise zero-cost proxies in order to speed-up AutoML. It will study a number of techniques in order to navigate large search spaces more effectively and increase fidelity of the zero-cost proxies, resulting in a significant improvements in both searching time and quality of discovered models, compared to the current state-of-the-art, making AutoML more efficient and sustainable.