Bayesian deep-learning prediction with sparse graphs

点击翻译按钮获取中文摘要

Many statistical problems involving images or relational data are described by relational processes. Among these are inference problems in functional magnetic resonance images (fMRI), social networks, and gene expression data. Also, recently, kernel-based sparse graphs have been introduced to explicitly model the partition data structure in Bayesian non-parametric (BNP) models, that otherwise was only induced as a side-effect of BNP models. Depending on the problem at hand, the likelihood, or the prior density of relational data may be modeled by sparse graph models. Some popular models are the Ising model, and the Potts models, in the discrete case; and Gibbs random fields, in the general case. The use of complex, realistic graph models to account for data relationships have the potential to dramatically improve the prediction of simple models such as multivariate linear regression. On the other hand, complex models such as deep-learning neural networks have proven to be able to predict well when no simpler model is known to work for very complex functional or high dimensional data, provided that the training data is chosen in a suitable manner. Usually large amounts of data are needed to fit the networks. We believe that good prediction can be achieve with less data if the network or other likelihood model is guided by auxiliary models that only use covariate information. Following the success of the semi-parametric Potts regression model, we propose to guide shallow and deep-learning by combining these type of complex predictors with a random partition model based on covariate proximity. The resulting model in the case of a neural network model is a semi-parametric Bayesian mixture neural network whose posterior predictions may be seen as averages over a large collection of possible partitions of the data. We aim at the development of methodologies for sparse-graphs in deep-learning, and of BNP models with explicit random partition structure driven by kernel-based sparse-graphs. The fitting and posterior prediction of the models may be done with Markov chain Monte Carlo methods. Though for the sake of efficiency, we proposed to realize approximate, but accurate, inference through variational methods, graph theory, and techniques derived from graph models in physics. Applications of particular interest in this project are inference for voxel activity in brain fMR images, and networks for prediction in social science.