Exploring Efficient Automated Design Choices for Robust Machine Learning Algorithms

探索稳健的机器学习算法的高效自动化设计选择

• 批准号: 2748823
• 金额: --
• 依托单位: University of Liverpool
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2022
• 资助国家: 英国
• 起止时间: 2022 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2748823
• 关键词: Exploring; Efficient; Automated; Design; Choices

Are you familiar with machine learning? Do you have an aptitude for analysing and disseminating information from a variety of outcomes? Would you like to assist GCHQ develop and design new algorithms for both time-efficiency and energy-efficiency solutions? Are you keen on developing novel approaches and using modern computing architectures that make it easy to apply Deep Learning and Gaussian Processes to real problems?Applying Machine Learning (ML) currently requires the data scientist to make design choices. These choices might relate, for example, to: choosing the number of layers and the number of neurons in each layer of a Deep Neural Network; choosing which kernel family to use in a Gaussian Process. Since ML algorithms often involve time-consuming training regimes, data scientists often find it laborious to iterate between (re)-identifying candidate design choices and (re)-training the ML algorithms. Furthermore, different design choices can alter both how many hyper-parameters (e.g. neuron weights or kernel widths and cross-covariance terms) need to considered but also how challenging it is to optimise the hyper-parameters of the ML algorithm. Since practitioners have limited time to perform sensitivity analyses with respect to these parameters, design choice are typically based on estimated performance (calculated as an average over the test set) with very limited, if any, consideration for the variance in this estimate. It is important that this variance is considered since it will determine how likely it is that performance on the test set will accurately predict empirical performance when the algorithm is deployed operationally. Indeed, robust performance requires that we do not optimise the hyper-parameters (e.g. using stochastic gradient descent) but generate a set of samples for the hyper-parameters that are consistent with the data and then average across these sampled values for the hyper-parameters.Numerical Bayesian algorithms exist that can explore the design choices and the possible hyper-parameter values associated with each design choice. Mature variants of these algorithms exist and involve the use of Markov-Chain Monte Carlo (MCMC), with Reversible Jump MCMC (RJMCMC) being a variant applicable in contexts where the design choice alters the number of hyper-parameters that need to be considered. In general, and particularly in the case of RJMCMC, these mature algorithms are sufficiently slow and computationally demanding that they are widely assumed to be impractical for practical use in real-world scenarios.Recent advances at the University of Liverpool have identified that Sequential Monte Carlo (SMC) samplers are an alternative family of numerical Bayesian algorithms that offer the potential to improve on both the time-efficiency and energy-efficiency of MCMC algorithms. In this context, SMC samplers can be considered to comprise a team of sub-algorithms that collaborate to explore the space of design choices and associated hyper-parameters. By distributing the sub-algorithms across parallel computational resources, SMC samplers can improve time-efficiency. Since the sub-algorithms only need to avoid all failing at once, they can each be more adventurous in their exploration than the single MCMC algorithm: this can lead to energy-efficiency gains. Perhaps surprisingly, the potential for SMC samplers to automate design choices, while also exploring the associated hyper-parameter values, is largely unexplored. This PhD will investigate the significant potential to apply SMC samplers in this context.

你熟悉机器学习吗？你有分析和传播来自各种结果的信息的能力吗？您是否愿意协助GCHQ为时间效率和能源效率解决方案开发和设计新的算法？您是否热衷于开发新的方法和使用现代计算体系结构，以便轻松地将深度学习和高斯过程应用于实际问题？应用机器学习(ML)目前需要数据科学家做出设计选择。例如，这些选择可能涉及：选择深度神经网络的层数和每层中的神经元数量；选择在高斯过程中使用哪个核心族。由于ML算法通常涉及耗时的训练机制，数据科学家经常发现在(重新)识别候选设计选择和(重新)训练ML算法之间迭代是很费力的。此外，不同的设计选择既可以改变需要考虑的超参数(例如，神经元权重或核宽度和互协方差项)的数量，也可以改变优化ML算法的超参数的挑战性。由于实践者对这些参数进行敏感性分析的时间有限，设计选择通常基于估计的性能(以测试集上的平均值计算)，而对此估计中的差异的考虑非常有限。重要的是要考虑这种差异，因为它将确定当算法部署到操作中时，测试集上的性能准确预测经验性能的可能性有多大。事实上，稳健的性能要求我们不优化超参数(例如使用随机梯度下降)，而是为与数据一致的超参数生成一组样本，然后对这些超参数的采样值进行平均。存在数值贝叶斯算法，可以探索设计选择以及与每个设计选择相关联的可能的超参数值。存在这些算法的成熟变体，并且涉及马尔可夫链蒙特卡罗(MCMC)的使用，其中可逆跳跃MCMC(RJMCMC)是一种变体，适用于设计选择改变需要考虑的超参数的数量的情况。总的来说，尤其是在RJMCMC的情况下，这些成熟的算法足够慢，并且对计算的要求很高，以至于它们被广泛地假设为不适用于现实世界的场景。利物浦大学最近的进展已经确认，序列蒙特卡罗(SMC)采样器是一种替代的数值贝叶斯算法家族，提供了提高MCMC算法的时间效率和能量效率的潜力。在这种情况下，SMC采样器可以被认为是由一组子算法组成的，这些子算法协作探索设计选择空间和相关的超参数。通过将子算法分布在并行计算资源上，SMC采样器可以提高时间效率。由于子算法只需要避免一次全部失败，因此它们在探索过程中可以比单一的MCMC算法更具冒险精神：这可以带来能源效率的提高。也许令人惊讶的是，SMC采样器在探索相关超参数值的同时，实现设计选择自动化的潜力在很大程度上还没有开发出来。本博士学位将探讨在此背景下应用SMC采样器的巨大潜力。