Data Reduction and Large-Scale Inference - Bayesian Coresets

贝叶斯方法在大规模数据环境中的使用是有吸引力的，由于它们提供的一致的不确定性量化和先验规范。不幸的是，贝叶斯推理算法通常不具有计算可扩展性，这使得它们在大型数据集上的应用变得困难或不可行。随着现代数据集不断变得越来越大，推理程序必须具有可扩展性，同时保留对其结果质量的理论保证。那么问题自然就出现了，如何以原则性的方式减少数据，以某种方式从海量的高维数据集中提取有意义的结构，并将其压缩成更小的低维数据集，分析成本更低。以前关于扩展贝叶斯推理的工作主要集中在增强算法上，例如，在每次迭代中只使用随机数据子样本。然而，通过利用数据通常是冗余的这一观点，最近关于贝叶斯核心集的工作提供了许多方法来找到比原始数据集小得多的数据的加权子集（称为核心集）。这种核心集，然后可以利用在许多现有的后验推理算法没有改变，提供计算加速和保证后验近似误差。通过确保核心集构造加上后续回归参数估计的组合成本小于从完整数据集估计推断参数的组合成本，可以实现显著的计算增益。这些想法也可以扩展到其他应用，例如贝叶斯推断，而不是使用点估计，参数是从后验分布使用MCMC或SMC技术进行采样。这样的采样过程涉及重复评估的似然函数，这是成本较低的使用一个小的核心集比它是为完整的数据集。关于这一主题的工作也可以采取的方向杂交coreset方法与非线性降维技术。这样的技术被设计为不减少数据点的数量，而是通过认识和利用数据可以集中在嵌入在高维空间中的低固有维度的流形周围的事实来减少每个数据点的维度。还有其他几个有趣的研究方向，其中的工作可能会采取;目前的coreset减少方法依赖于数据点的完全或有条件的独立。这些方法在多大程度上可以扩展到这一制度之外？降维方法是否可以放在一个有充分根据的统一概率框架内？大学项目主管将是尼克·怀特利和罗伯特·艾利森。“工业”联合主管将来自NCSC内的机器学习研究小组，该小组全面参与大规模贝叶斯推理技术的研究，包括数据简化方法，并将加入我们定期进行的详细技术讨论。该小组在数据科学/计算统计/机器学习领域与英国大学研究界以及Alan Turing Institute和NCSC研究活动有着良好的联系

The use of Bayesian methods in large-scale data settings is attractive due to the coherent uncertainty quantification, and prior specification they provide. Unfortunately, Bayesian inference algorithms are not generally computationally scalable, making their application to large datasets difficult or infeasible. As modern data sets continue to grow ever larger, it is essential for inference procedures to be scalable whilst retaining theoretical guarantees on the quality of their results. The question then naturally arises of how to reduce data in a principled manner, somehow extracting the meaningful structure in massive, high-dimensional data sets and condensing it into a smaller, lower-dimensional data sets which are less costly to analyse. Previous work on scaling Bayesian inference has focused on augmenting algorithms to, for example, use only a random data subsample at each iteration. However, by leveraging the insight that data is often redundant, recent work on Bayesian coresets has provided numerous approaches to finding a weighted subset of the data (called a coreset) that is much smaller than the original dataset. This coreset can then be exploited in many existing posterior inference algorithms without alteration, providing computational speedup and guarantees on posterior approximation error. Significant computational gains can be achieved by ensuring that the combined cost of coreset construction plus follow-on regression-parameter estimation from the coreset is less than that of estimating the inference parameters from the full dataset. These ideas can extend to other applications too, for example to Bayesian inference where, rather than using point estimates, parameters are sampled from a posterior distribution using MCMC or SMC techniques. Such sampling processes involve repeatedly evaluating the likelihood function which is less costly using a small coreset than it is for the full dataset. Work on this topic could also be taken in the direction hybridizing coreset methods with nonlinear dimensionality reduction techniques. Such techniques are designed not to reduce the number of data points, but rather the dimension of each data point, by recognizing and exploiting the fact that data may be concentrated around a manifold of low intrinsic dimension, embedded in a high-dimensional space. There are several other interesting research directions in which the work might be taken; current coreset reduction methods rely on full or conditional independence of data points. To what extent can the methods be extended beyond this regime? Can dimensionality reduction methods be placed within a well-founded and unified probabilistic framework? The University project supervisors will be Nick Whiteley and Robert Allison. "Industrial" co-supervisor(s) will be from the machine learning research group within the NCSC which is fully engaged on research into largescale Bayesian inference techniques, including data-reduction methods, and will join with our regular detailed technical discussions. This group is well connected across the UK university research community in the areas of data-science/computational-statistics/machine-learning as well as with the Alan Turing Institute and with NCSC research activities