Expectation Propagation for t-Exponential Family Using q-Algebra
Expectation Propagation for t-Exponential Family Using q-Algebra
复制标题
DOI：
--
复制
发表时间：
2017-05
期刊：
影响因子：
--
通讯作者：
Futoshi Futami;Issei Sato;Masashi Sugiyama
中科院分区：
文献类型：
--
作者：
Futoshi Futami;Issei Sato;Masashi Sugiyama
指数族分布在机器学习中非常有用，因为它们的计算可以通过自然参数有效地执行。指数族最近被扩展到t指数族，它包含学生-t分布作为族成员，从而使我们能够很好地处理有噪声的数据。然而，由于t指数族被变形的指数拒绝，因此我们不能推导出t指数族的有效学习算法，如期望传播(EP)。本文借用统计物理学中Q-代数的数学工具，证明了分布的伪可加性使我们可以通过自然参数来计算t-指数族分布。然后，我们提出了一种t指数族的期望传播(EP)算法，它通过简单的矩匹配提供对后验分布或预测分布的确定性近似。最后，我们将所提出的EP算法应用于贝叶斯点机和学生-t过程分类，并用数值方法证明了它们的性能。
Exponential family distributions are highly useful in machine learning since their calculation can be performed efficiently through natural parameters. The exponential family has recently been extended to the t-exponential family, which contains Student-t distributions as family members and thus allows us to handle noisy data well. However, since the t-exponential family is denied by the deformed exponential, we cannot derive an efficient learning algorithm for the t-exponential family such as expectation propagation (EP). In this paper, we borrow the mathematical tools of q-algebra from statistical physics and show that the pseudo additivity of distributions allows us to perform calculation of t-exponential family distributions through natural parameters. We then develop an expectation propagation (EP) algorithm for the t-exponential family, which provides a deterministic approximation to the posterior or predictive distribution with simple moment matching. We finally apply the proposed EP algorithm to the Bayes point machine and Student-t process classication, and demonstrate their performance numerically.