Poisson Dependency Networks: Gradient Boosted Models for Multivariate Count Data
Poisson Dependency Networks: Gradient Boosted Models for Multivariate Count Data
复制标题
DOI：
10.1007/s10994-015-5506-z
复制
发表时间：
2015-09
期刊：
影响因子：
7.5
通讯作者：
Fabian Hadiji;Alejandro Molina;Sriraam Natarajan;K. Kersting
中科院分区：
文献类型：
--
作者：
Fabian Hadiji;Alejandro Molina;Sriraam Natarajan;K. Kersting
虽然计数数据越来越普遍，令人惊讶的是，很少有工作采用概率图形模型建模计数数据。事实上，单变量的情况下已经得到了很好的研究，但是，在许多情况下，计数相互影响，不应该被认为是独立的。标准的图形模型，如多项式或高斯模型，也经常不适合，因为它们忽略了自然数的无限范围或计数变量分布的潜在不对称形状。现有的Poisson图模型只能模拟负的条件依赖关系，或者忽略计数的预测，或者不能很好地扩展。为了简化多元计数数据的建模，我们引入了一系列新颖的Poisson图模型，称为Poisson依赖网络（PDNs）。PDN由一组局部条件泊松分布组成，每个分布表示单个计数变量在给定其他变量的情况下的概率，这自然有助于简单的吉布斯采样推断。与现有的泊松图模型相比，PDN是非参数的，并且使用函数梯度上升来训练，即，增强泊松分布的特别简单的形式允许我们开发第一种乘法提升方法：从初始常数值开始，或者是对数线性泊松模型或泊松回归树，PDN表示为逐步优化中生长的回归模型的乘积。我们在几个真实的世界数据集上证明，PDN可以对正相关和负相关进行建模，并且可以很好地扩展，同时通常优于最先进的技术，特别是在使用乘法更新时。
Although count data are increasingly ubiquitous, surprisingly little work has employed probabilistic graphical models for modeling count data. Indeed the univariate case has been well studied, however, in many situations counts influence each other and should not be considered independently. Standard graphical models such as multinomial or Gaussian ones are also often ill-suited, too, since they disregard either the infinite range over the natural numbers or the potentially asymmetric shape of the distribution of count variables. Existing classes of Poisson graphical models can only model negative conditional dependencies or neglect the prediction of counts or do not scale well. To ease the modeling of multivariate count data, we therefore introduce a novel family of Poisson graphical models, called Poisson Dependency Networks (PDNs). A PDN consists of a set of local conditional Poisson distributions, each representing the probability of a single count variable given the others, that naturally facilitates a simple Gibbs sampling inference. In contrast to existing Poisson graphical models, PDNs are non-parametric and trained using functional gradient ascent, i.e., boosting. The particularly simple form of the Poisson distribution allows us to develop the first multiplicative boosting approach: starting from an initial constant value, alternatively a log-linear Poisson model, or a Poisson regression tree, a PDN is represented as products of regression models grown in a stage-wise optimization. We demonstrate on several real world datasets that PDNs can model positive and negative dependencies and scale well while often outperforming state-of-the-art, in particular when using multiplicative updates.