Neural predictive monitoring and a comparison of frequentist and Bayesian approaches
Neural predictive monitoring and a comparison of frequentist and Bayesian approaches
复制标题
DOI：
10.1007/s10009-021-00623-1
复制
发表时间：
2021-05
影响因子：
1.5
通讯作者：
L. Bortolussi;Francesca Cairoli;Nicola Paoletti;S. Smolka;S. Stoller
中科院分区：
文献类型：
--
作者：
L. Bortolussi;Francesca Cairoli;Nicola Paoletti;S. Smolka;S. Stoller
神经状态分类(NSC)是最近提出的一种使用深度神经网络(DNN)对混合自动机(HA)进行运行时预测监控的方法。NSC将DNN训练为近似可达性预测器，如果在给定时间范围内可从x到达不安全状态，则将HA状态暴露标记为肯定，否则将x标记为否定。NSC预报器具有非常高的精度，但容易出现预测误差，从而对可靠性产生负面影响。为了克服这一局限性，我们提出了神经预测监测(NPM)，这是一种用预测不确定性的估计来补充NSC预测的技术。这些措施为拒绝可能不正确的预测提供了原则性标准，而不知道真正的可达性值。我们还提出了一种主动学习方法，显著降低了NSC预测器的错误率和拒绝预测的比例。我们开发了两个版本的NPM，分别基于频率和贝叶斯技术来学习预测器和拒绝规则。这两个版本都非常高效，计算时间在毫秒量级，并且有效，在我们的实验评估中成功地拒绝了几乎所有错误的预测。在我们对六个混合系统的基准测试套件上的实验中，我们发现频率法的性能始终优于贝叶斯方法。我们还观察到，贝叶斯方法不太实用，需要仔细选择特定于问题的超参数。
Neural state classification (NSC) is a recently proposed method for runtime predictive monitoring of hybrid automata (HA) using deep neural networks (DNNs). NSC trains a DNN as an approximatereachability predictorthat labels an HA statexaspositiveif an unsafe state is reachable fromxwithin a given time bound, and labelsxasnegativeotherwise. NSC predictors have very high accuracy, yet are prone to prediction errors that can negatively impact reliability. To overcome this limitation, we presentneural predictive monitoring(NPM), a technique that complements NSC predictions with estimates of the predictive uncertainty. These measures yield principled criteria for the rejection of predictions likely to be incorrect, without knowing the true reachability values. We also present an active learning method that significantly reduces the NSC predictor’s error rate and the percentage of rejected predictions. We develop two versions of NPM based, respectively, on the use of frequentist and Bayesian techniques to learn the predictor and the rejection rule. Both versions are highly efficient, with computation times on the order of milliseconds, and effective, managing in our experimental evaluation to successfully reject almost all incorrect predictions. In our experiments on a benchmark suite of six hybrid systems, we found that the frequentist approach consistently outperforms the Bayesian one. We also observed that the Bayesian approach is less practical, requiring a careful and problem-specific choice of hyperparameters.