Bayesian deep-learning prediction with sparse graphs

稀疏图的贝叶斯深度学习预测

• 批准号: RGPIN-2019-05444
• 负责人: Murua, Alejandro
• 金额: $ 1.82万
• 依托单位: Université de Montréal
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=680321
• 关键词: Bayesian; deep; learning; prediction; sparse

Many statistical problems involving images or relational data are described by relational processes. Among these are inference problems in functional magnetic resonance images (fMRI), social networks, and gene expression data. Also, recently, kernel-based sparse graphs have been introduced to explicitly model the partition data structure in Bayesian non-parametric (BNP) models, that otherwise was only induced as a side-effect of BNP models. Depending on the problem at hand, the likelihood, or the prior density of relational data may be modeled by sparse graph models. Some popular models are the Ising model, and the Potts models, in the discrete case; and Gibbs random fields, in the general case.******The use of complex, realistic graph models to account for data relationships have the potential to dramatically improve the prediction of simple models such as multivariate linear regression. On the other hand, complex models such as deep-learning neural networks have proven to be able to predict well when no simpler model is known to work for very complex functional or high dimensional data, provided that the training data is chosen in a suitable manner. Usually large amounts of data are needed to fit the networks. We believe that good prediction can be achieve with less data if the network or other likelihood model is guided by auxiliary models that only use covariate information. Following the success of the semi-parametric Potts regression model, we propose to guide shallow and deep-learning by combining these type of complex predictors with a random partition model based on covariate proximity. The resulting model in the case of a neural network model is a semi-parametric Bayesian mixture neural network whose posterior predictions may be seen as averages over a large collection of possible partitions of the data.******We aim at the development of methodologies for sparse-graphs in deep-learning, and of BNP models with explicit random partition structure driven by kernel-based sparse-graphs. The fitting and posterior prediction of the models may be done with Markov chain Monte Carlo methods. Though for the sake of efficiency, we proposed to realize approximate, but accurate, inference through variational methods, graph theory, and techniques derived from graph models in physics.******Applications of particular interest in this project are inference for voxel activity in brain fMR images, and networks for prediction in social science.

许多涉及图像或关系数据的统计问题都用关系过程来描述。其中包括功能磁共振图像（fMRI）、社会网络和基因表达数据的推理问题。此外，最近，基于核的稀疏图已经被引入贝叶斯非参数（BNP）模型来显式地建模分区数据结构，否则只能作为BNP模型的副作用引起。根据手头的问题，关系数据的可能性或先验密度可以通过稀疏图模型建模。一些流行的模型是离散情况下的Ising模型和Potts模型；和吉布斯随机场，一般情况下。******使用复杂的、真实的图形模型来解释数据关系，有可能极大地改善简单模型（如多元线性回归）的预测。另一方面，深度学习神经网络等复杂模型已经被证明能够很好地预测，当已知没有更简单的模型可以用于非常复杂的功能或高维数据时，只要以合适的方式选择训练数据。通常需要大量的数据来拟合网络。我们认为，如果网络或其他似然模型由仅使用协变量信息的辅助模型指导，则可以用较少的数据实现良好的预测。随着半参数Potts回归模型的成功，我们提出通过将这些类型的复杂预测因子与基于协变量接近性的随机划分模型相结合来指导浅层和深度学习。在神经网络模型的情况下，得到的模型是一个半参数贝叶斯混合神经网络，其后验预测可以看作是大量可能分区数据的平均值。******我们的目标是开发深度学习中的稀疏图方法，以及由基于核的稀疏图驱动的具有显式随机划分结构的BNP模型。模型的拟合和后验预测可以用马尔可夫链蒙特卡罗方法来完成。虽然出于效率的考虑，我们提出通过变分方法、图论和源自物理图模型的技术来实现近似但准确的推理。******在这个项目中特别感兴趣的应用是脑fMR图像中体素活动的推理，以及社会科学预测的网络。