Scalable Online Machine Learning

该项目涉及扩展最先进的技术，使机器学习能够充分利用永无止境的数据流中的信息。随着时间的推移，额外的数据包含了有助于改进机器学习的信息。不使用这些信息会产生一致但令人惊讶的错误：这通常发生在训练数据相对于算法的经验较小时。概念漂移也可能发生：时间的推移也为导致数据变化的现象提供了空间。概念漂移的结果是，即使感兴趣的现象没有改变，由于统计环境的变化，机器学习的性能也容易下降。此外，由于历史数据的数量不断增长，给定有限的数据存储和计算资源，需要创新的技术来总结数据中存在的信息，而不需要存储所有接收到的原始数据。首先，为了减少存储和计算，否则将需要，相关的数据接收到的当前时间将总结在一个自适应的基于树的数据结构。这种数据结构的定义将建立在以前的工作近似贝叶斯计算，并涉及近似的信息，在原始数据中的摘要。其次，为了确保概念漂移得到照顾，这些摘要将明确涉及机器学习试图估计的参数的时间导数。最后，为了最大限度地提高性能，以前涉及变分推理的相关工作，将扩展到考虑上述数据结构，也将适用于考虑数值贝叶斯推理。该方法将应用于现实世界的数据集，涉及以下组合：（例如与感兴趣的罕见事件有关）;在长时间尺度上平稳波动的参数（例如迷因的扩散传播）;概念的突然转变（例如新迷因的出现）。预计此类数据集将涉及大型且不断增长的文本语料库（例如社交媒体）

The project relates to extending the state-of-the-art to enable machine learning to fully capitalise on the information present in never-ending data streams. The additional data that arrives over time contains information that should facilitate improved machine learning. Not using this information gives rise to consistent yet surprising errors: this typically occurs when the training data is small relative to the algorithm's empirical experience. Concept drift can also occur: the passage of time also provides scope for the phenomena that give rise to the data to change. The result of concept drift is that, even if the phenomena of interest do not change, because the statistical environment changes, the performance of the machine learning is prone to degrading. Furthermore, since the quantity of historic data is ever growing, given finite data storage and computational resources, innovative techniques are needed to summarise the information present in data and currently pertinent without requiring all the raw data ever received to be stored.The proposed solution involves three novel components. First, to reduce the storage and computation that would otherwise be required, the pertinent data received up to the current time will be summarised in an adaptive tree-based data structure. This definition of this data structure will build on previous work on Approximate Bayesian Computation and involve approximating the information present in the raw data with summaries. Second, to ensure concept drift is catered for, these summaries will explicitly relate to the time-derivatives of the parameters that the machine learning is attempting to estimate. Finally, to maximise performance, previous related work involving variational inference, which will be extended to consider the aforementioned data structures, will also be adapted to consider numerical Bayesian inference.The approach will be applied to real-world datasets involving combinations of: near-constant parameters for which concept drift is not relevant (e.g. related to rare events of interest); parameters that fluctuate smoothly over long timescales (e.g. diffusive spread of memes); sudden shifts in concepts (e.g. new memes appearing). Such datasets are anticipated to involve large and continually growing text corpuses (e.g. social media)