Improving efficiency and sustainability of automated machine learning

近年来，自动机器学习（AutoML）已被证明在发现有效的神经网络方面取得了成功，在许多机器学习任务中改进了手动设计的网络-例如图像分类，超分辨率，自动语音识别等。然而，早期的AutoML技术对计算资源的要求非常高，通常需要比正常训练多几个数量级的GPU时间。尽管自那时以来，AutoML在提高效率方面取得了重大改进，但自动优化神经网络结构所需的时间和资源仍然是限制AutoML适用性的主要因素之一，特别是在考虑非常大的搜索空间和数据集时。该项目旨在研究现有方法的可扩展性和相关限制，特别关注那些利用零成本代理来加速AutoML的方法。它将研究许多技术，以便更有效地导航大型搜索空间，并提高零成本代理的保真度，从而与当前最先进的技术相比，显着改善搜索时间和发现模型的质量，使AutoML更有效和可持续。

In the recent years, automated machine learning (AutoML) has been proven successful in discovering efficient neural networks, improving upon manually designed networks in a number of machine learning tasks - such as image classification, super-resolution, automatic speech recognition, and more. However, early AutoML techniques were extremely demanding in terms of computational resources, often necessitating orders of magnitude more GPU-hours than a normal training. Although significant improvements have been made in making AutoML more efficient since then, the time and resources required to automatically optimise a neural network's structure still remain one of the main factors limiting applicability of AutoML, especially when very large search spaces and datasets are considered. This project aims to investigate the scalability and related limitations of the existing approaches with a special focus on those that utilise zero-cost proxies in order to speed-up AutoML. It will study a number of techniques in order to navigate large search spaces more effectively and increase fidelity of the zero-cost proxies, resulting in a significant improvements in both searching time and quality of discovered models, compared to the current state-of-the-art, making AutoML more efficient and sustainable.