Bayesian deep-learning prediction with sparse graphs

许多涉及图像或关系数据的统计问题都是用关系过程来描述的。其中包括功能磁共振成像(FMRI)、社交网络和基因表达数据中的推理问题。最近，基于核的稀疏图被引入来显式地建模贝叶斯非参数(BNP)模型中的划分数据结构，否则这只是BNP模型的副作用。根据手头的问题，关系数据的可能性或先验密度可能会用稀疏图模型来建模。一些流行的模型是离散情况下的Ising模型和Potts模型；以及一般情况下的Gibbs随机场。*使用复杂、现实的图形模型来说明数据关系，有可能显著提高多元线性回归等简单模型的预测能力。另一方面，如果以适当的方式选择训练数据，当已知没有更简单的模型对非常复杂的函数或高维数据起作用时，诸如深度学习神经网络的复杂模型被证明能够很好地预测。通常需要大量数据来适应网络。我们认为，如果网络或其他似然模型由仅使用协变量信息的辅助模型指导，则可以用较少的数据实现良好的预测。继半参数Potts回归模型的成功之后，我们建议通过将这些类型的复杂预测因子与基于协变量邻近的随机划分模型相结合来指导浅层和深层学习。在神经网络模型的情况下，得到的模型是一个半参数贝叶斯混合神经网络，其后验预测可以被视为大量可能的数据分区的平均值。*我们的目标是发展深度学习中的稀疏图的方法，以及由基于核的稀疏图驱动的具有显式随机分区结构的BNP模型。模型的拟合和后验预测可用马尔科夫链蒙特卡罗方法进行。虽然为了提高效率，我们提出了通过变分方法、图论和源自物理图形模型的技术来实现近似但准确的推理。*这个项目特别感兴趣的应用是对大脑FMR图像中的体素活动进行推理，以及在社会科学中用于预测的网络。

Many statistical problems involving images or relational data are described by relational processes. Among these are inference problems in functional magnetic resonance images (fMRI), social networks, and gene expression data. Also, recently, kernel-based sparse graphs have been introduced to explicitly model the partition data structure in Bayesian non-parametric (BNP) models, that otherwise was only induced as a side-effect of BNP models. Depending on the problem at hand, the likelihood, or the prior density of relational data may be modeled by sparse graph models. Some popular models are the Ising model, and the Potts models, in the discrete case; and Gibbs random fields, in the general case.******The use of complex, realistic graph models to account for data relationships have the potential to dramatically improve the prediction of simple models such as multivariate linear regression. On the other hand, complex models such as deep-learning neural networks have proven to be able to predict well when no simpler model is known to work for very complex functional or high dimensional data, provided that the training data is chosen in a suitable manner. Usually large amounts of data are needed to fit the networks. We believe that good prediction can be achieve with less data if the network or other likelihood model is guided by auxiliary models that only use covariate information. Following the success of the semi-parametric Potts regression model, we propose to guide shallow and deep-learning by combining these type of complex predictors with a random partition model based on covariate proximity. The resulting model in the case of a neural network model is a semi-parametric Bayesian mixture neural network whose posterior predictions may be seen as averages over a large collection of possible partitions of the data.******We aim at the development of methodologies for sparse-graphs in deep-learning, and of BNP models with explicit random partition structure driven by kernel-based sparse-graphs. The fitting and posterior prediction of the models may be done with Markov chain Monte Carlo methods. Though for the sake of efficiency, we proposed to realize approximate, but accurate, inference through variational methods, graph theory, and techniques derived from graph models in physics.******Applications of particular interest in this project are inference for voxel activity in brain fMR images, and networks for prediction in social science.