Scalable Online Machine Learning

可扩展的在线机器学习

• 批准号: 2599529
• 金额: --
• 依托单位: University of Liverpool
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2021
• 资助国家: 英国
• 起止时间: 2021 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2599529
• 关键词: Scalable; Online; Machine; Learning

Many of the current problems within the modern machine learning sector involve dealing with ever growing datasets where accurately and efficiently estimating a posterior distribution to make predictions is very difficult due to being able to identify the correct information from datasets which is informative to modelling our target distribution. As well as volume, these models often increase in dimensionality (and therefore complexity) which current common methods such as Markov Chain Monte Carlo (MCMC)- sampling struggle to do efficiently, as they become much slower as the complexity of the model increases. Other types of sampling methods though have shown to scale much better than traditional MCMC. For example, Sequential Monte Carlo sampling (SMC) and Hamiltonian Monte Carlo (HMC) sampling scale a lot better with dimensionality. These techniques are nowhere near as well researched though and have their own problems, so improving upon these further by introducing Reversible Jump (RJ) MCMC and Mass Matrices (MM) will be a main stay of my research. My colleague Josh Murphy will be working on Concept Drift, and I will be working on Eternal Learning. Initially there may be some similarities on the research we undertake but due to the differences in problem concepts, this will start to diverge within the first year. However, further down the line there will likely be some collaboration as the methods developed in each area may have some overlap depending on the specific problem we are undertaking. Therefore, I will also be researching methods to compress data from a constant and endless source but so that little to no information is lost during our inference with the aforementioned sampling methods. This will need to be done, as for eternal learning, the first sample will be just as important as the most recent one. Finally, these new methods will be applied to a Bayesian deep learning context. In neural networks, backpropagation for use in calculating the weights and biases of models is very computationally expensive. Early research with using particle filters as an alternative (or in an ensemble method) to backpropagation has recently started in the past couple of years as they are less computationally expensive. I will expand upon this by implementing a quasi-differentiable SMC sampler (as opposed to a particle filter) to aid the optimization process in neural networks.

现代机器学习领域目前的许多问题都涉及到处理不断增长的数据集，在这些数据集中，准确有效地估计后验分布来进行预测是非常困难的，因为能够从数据集中识别正确的信息，这些信息对我们的目标分布建模是有用的。除了体积之外，这些模型通常会增加维度（因此也增加了复杂性），而目前常用的方法，如马尔可夫链蒙特卡罗（MCMC）采样，很难有效地做到这一点，因为随着模型复杂性的增加，它们会变得慢得多。其他类型的采样方法已经证明比传统的MCMC更好地扩展。例如，序贯蒙特卡罗采样（SMC）和汉密尔顿蒙特卡罗（HMC）采样在维度上的缩放效果要好得多。虽然这些技术还没有得到很好的研究，并且有自己的问题，所以通过引入可逆跳跃（RJ）MCMC和质量矩阵（MM）来进一步改进这些技术将是我研究的主要内容。我的同事Josh Murphy将致力于概念漂移，我将致力于永恒的学习。最初，我们进行的研究可能有一些相似之处，但由于问题概念的差异，这将在第一年内开始出现分歧。然而，随着我们正在处理的具体问题的不同，在每个领域开发的方法可能会有一些重叠，因此可能会有一些合作。因此，我也将研究从恒定和无尽的来源压缩数据的方法，但在我们使用上述采样方法进行推断的过程中，几乎没有信息丢失。这将需要完成，因为对于永恒的学习，第一个样本将与最近的样本一样重要。最后，这些新方法将应用于贝叶斯深度学习环境。在神经网络中，用于计算模型的权重和偏差的反向传播在计算上非常昂贵。使用粒子滤波器作为反向传播的替代方法（或集成方法）的早期研究最近在过去几年开始，因为它们的计算成本较低。我将通过实现准可微SMC采样器（与粒子滤波器相反）来扩展这一点，以帮助神经网络中的优化过程。