Guaranteed approximation error estimation of neural networks and model modification
Guaranteed approximation error estimation of neural networks and model modification
复制标题
DOI：
10.1016/j.neunet.2022.03.023
复制
发表时间：
2022-03
期刊：
影响因子：
--
通讯作者：
Yejiang Yang;Tao Wang;Jefferson P. Woolard;Weiming Xiang
中科院分区：
文献类型：
--
作者：
Yejiang Yang;Tao Wang;Jefferson P. Woolard;Weiming Xiang
逼近误差是神经网络模型验证过程中的一个重要指标。本文讨论了神经网络的保误差估计问题及其在保系统建模和保神经网络压缩中的应用。首先，提出了前馈神经网络保误差估计的概念，它旨在提供训练神经网络相对于本质上包含无穷多个值的紧致输入集的最坏情况下的逼近误差。针对原系统不同的先验信息，提出了两种计算逼近误差上界的方法：Lipschitz常数分析方法和集值可达性分析方法。基于保证逼近误差估计框架，提出了一种从数据集中获取参数值的优化方法。给出了机械臂和神经网络的压缩实例，说明了该方法的有效性。
Approximation error is a key measure in the process of model validation and verification for neural networks. In this paper, the problems of guaranteed error estimation of neural networks and applications to assured system modeling and assured neural network compression are addressed. First, a concept called guaranteed error estimation of feedforward neural networks is proposed, which intends to provide the worst-case approximation error of a trained neural network with respect to a compact input set essentially containing an infinite number of values. Given different prior information about the original system, two approaches including Lipschitz constant analysis and set-valued reachability analysis methods are developed to efficiently compute upper-bounds of approximation errors. Based on the guaranteed approximation error estimation framework, an optimization for obtaining parameter values from data set is proposed. A robotic arm and neural network compression examples are presented to illustrate the effectiveness of our approach.