Hierarchical Bayesian-Kalman models for regularisation and ARD in sequential learning
Hierarchical Bayesian-Kalman models for regularisation and ARD in sequential learning
复制标题
DOI：
--
复制
发表时间：
1997-09
期刊：
影响因子：
--
通讯作者：
J. Freitas;M. Niranjan;A. Gee
中科院分区：
文献类型：
--
作者：
J. Freitas;M. Niranjan;A. Gee
在本文中，我们表明，层次贝叶斯建模方法的顺序学习导致许多有趣的属性，如正则化和自动相关性确定。我们确定在这个层次结构中的三个推理水平，即模型选择，参数估计和噪声估计。在数据顺序到达的环境中，诸如交叉验证之类的技术来实现正则化或模型选择是不可能的。贝叶斯方法，扩展卡尔曼滤波在参数估计水平，允许正则化的最小方差框架内。一个多层感知器被用来产生扩展卡尔曼滤波器的非线性测量映射。我们描述了几种算法在噪声估计水平，这使我们能够实现自适应正则化和自动相关性确定的模型输入和基函数。本文的一个重要贡献是展示了扩展卡尔曼滤波中的自适应噪声估计、多个自适应学习率和多个平滑正则化系数之间的理论联系。我
In this paper, we show that a hierarchical Bayesian modelling approach to sequential learning leads to many interesting attributes such as regularisation and automatic relevance determination. We identify three inference levels within this hierarchy, namely model selection, parameter estimation and noise estimation. In environments where data arrives sequentially, techniques such as cross-validation to achieve regularisation or model selection are not possible. The Bayesian approach, with extended Kalman ltering at the parameter estimation level, allows for regularisation within a minimum variance framework. A multi-layer perceptron is used to generate the extended Kalman lter nonlinear measurements mapping. We describe several algorithms at the noise estimation level, which allow us to implement adaptive regularisation and automatic relevance determination of model inputs and basis functions. An important contribution of this paper is to show the theoretical links between adaptive noise estimation in extended Kalman ltering, multiple adaptive learning rates and multiple smoothing regularisation coe cients. i