Data Reduction and Large-Scale Inference - Bayesian Coresets

点击翻译按钮获取中文摘要

The use of Bayesian methods in large-scale data settings is attractive due to the coherent uncertainty quantification, and prior specification they provide. Unfortunately, Bayesian inference algorithms are not generally computationally scalable, making their application to large datasets difficult or infeasible. As modern data sets continue to grow ever larger, it is essential for inference procedures to be scalable whilst retaining theoretical guarantees on the quality of their results. The question then naturally arises of how to reduce data in a principled manner, somehow extracting the meaningful structure in massive, high-dimensional data sets and condensing it into a smaller, lower-dimensional data sets which are less costly to analyse. Previous work on scaling Bayesian inference has focused on augmenting algorithms to, for example, use only a random data subsample at each iteration. However, by leveraging the insight that data is often redundant, recent work on Bayesian coresets has provided numerous approaches to finding a weighted subset of the data (called a coreset) that is much smaller than the original dataset. This coreset can then be exploited in many existing posterior inference algorithms without alteration, providing computational speedup and guarantees on posterior approximation error. Significant computational gains can be achieved by ensuring that the combined cost of coreset construction plus follow-on regression-parameter estimation from the coreset is less than that of estimating the inference parameters from the full dataset. These ideas can extend to other applications too, for example to Bayesian inference where, rather than using point estimates, parameters are sampled from a posterior distribution using MCMC or SMC techniques. Such sampling processes involve repeatedly evaluating the likelihood function which is less costly using a small coreset than it is for the full dataset. Work on this topic could also be taken in the direction hybridizing coreset methods with nonlinear dimensionality reduction techniques. Such techniques are designed not to reduce the number of data points, but rather the dimension of each data point, by recognizing and exploiting the fact that data may be concentrated around a manifold of low intrinsic dimension, embedded in a high-dimensional space. There are several other interesting research directions in which the work might be taken; current coreset reduction methods rely on full or conditional independence of data points. To what extent can the methods be extended beyond this regime? Can dimensionality reduction methods be placed within a well-founded and unified probabilistic framework? The University project supervisors will be Nick Whiteley and Robert Allison. "Industrial" co-supervisor(s) will be from the machine learning research group within the NCSC which is fully engaged on research into largescale Bayesian inference techniques, including data-reduction methods, and will join with our regular detailed technical discussions. This group is well connected across the UK university research community in the areas of data-science/computational-statistics/machine-learning as well as with the Alan Turing Institute and with NCSC research activities