Bayesian Random Vector Functional-Link Networks for Robust Data Modeling
Bayesian Random Vector Functional-Link Networks for Robust Data Modeling
复制标题
DOI：
10.1109/tcyb.2017.2726143
复制
发表时间：
2018-07
影响因子：
11.8
通讯作者：
Simone Scardapane;Dianhui Wang;A. Uncini
中科院分区：
文献类型：
--
作者：
Simone Scardapane;Dianhui Wang;A. Uncini
随机向量函数链接（RVFL）网络是一种具有单个隐层和线性输出层的随机多层感知器，可以通过求解线性建模问题来训练。特别是，它们通常使用（正则化）最小二乘方法的封闭形式解进行训练。本文介绍了几种可供选择的策略，执行完全贝叶斯推理（BI）的RVFL网络。与标准或经典方法不同，我们提出的贝叶斯训练算法允许在网络的最佳输出权重上导出整个概率分布，而不是根据某些给定标准（例如，最小二乘）。这提供了几个已知的优点，包括在训练过程中引入额外的先验知识的可能性，在测试阶段期间的不确定性测量的可用性，以及从给定数据自动推断超参数的能力。在本文中，两个BI算法的回归首先提出，在一些实际的假设下，可以实现一个简单的迭代过程与封闭形式的计算。仿真结果表明，所提出的算法之一，贝叶斯RVFL，是能够优于标准的RVFL网络的训练算法，通过线搜索过程中仔细选择适当的正则化因子。基于变分推理的一般策略，也提出了一个应用程序的数据建模问题的噪声输出或离群值。正如我们在本文中讨论的那样，使用自动区分的最新进展，这种策略可以立即应用于广泛的其他情况。
Random vector functional-link (RVFL) networks are randomized multilayer perceptrons with a single hidden layer and a linear output layer, which can be trained by solving a linear modeling problem. In particular, they are generally trained using a closed-form solution of the (regularized) least-squares approach. This paper introduces several alternative strategies for performing full Bayesian inference (BI) of RVFL networks. Distinct from standard or classical approaches, our proposed Bayesian training algorithms allow to derive an entire probability distribution over the optimal output weights of the network, instead of a single pointwise estimate according to some given criterion (e.g., least-squares). This provides several known advantages, including the possibility of introducing additional prior knowledge in the training process, the availability of an uncertainty measure during the test phase, and the capability of automatically inferring hyper-parameters from given data. In this paper, two BI algorithms for regression are first proposed that, under some practical assumptions, can be implemented by a simple iterative process with closed-form computations. Simulation results show that one of the proposed algorithms, Bayesian RVFL, is able to outperform standard training algorithms for RVFL networks with a proper regularization factor selected carefully via a line search procedure. A general strategy based on variational inference is also presented, with an application to data modeling problems with noisy outputs or outliers. As we discuss in this paper, using recent advances in automatic differentiation this strategy can be applied to a wide range of additional situations in an immediate fashion.