Deep State Inference: Toward Behavioral Model Inference of Black-Box Software Systems
Deep State Inference: Toward Behavioral Model Inference of Black-Box Software Systems
复制标题
DOI：
10.1109/tse.2021.3128820
复制
发表时间：
2021-01
影响因子：
7.4
通讯作者：
Foozhan Ataiefard;Mohammad Jafar Mashhadi;H. Hemmati;Neil Walkinshaw
中科院分区：
文献类型：
--
作者：
Foozhan Ataiefard;Mohammad Jafar Mashhadi;H. Hemmati;Neil Walkinshaw
许多软件工程任务，例如测试、调试和异常检测，都可以从推断软件行为模型的能力中受益。大多数现有的推理方法都假设访问代码来收集执行序列。在本文中，我们研究了一个黑盒场景，其中被分析的系统无法以这种方式进行检测。在分析控制系统日志时，这种情况尤其常见，这些日志通常采用连续信号的形式。在这种情况下，执行轨迹相当于输入和输出信号的多元时间序列，其中系统的不同状态对应于时间序列中的不同“阶段”。从推理的角度来看，挑战在于检测这些相变何时发生。不幸的是，大多数现有的解决方案要么是单变量的，要么对数据分布做出假设，要么学习能力有限。在本文中，我们提出了一种混合深度神经网络，它接受多元时间序列作为输入，并应用一组卷积层和循环层来学习信号和模式之间随时间的非线性相关性。我们展示了如何使用这种方法来准确检测状态变化，以及如何将推断的模型成功应用于迁移学习场景，以准确处理来自具有相似执行特征的不同产品的跟踪。我们对两个无人机自动驾驶案例研究（一个工业和一个开源）的实验结果表明，我们的方法非常准确（状态分类的 F1 分数超过 90%），并且显着改善了基线（变化点检测高达 102%）。通过使用迁移学习，我们还表明，开源案例研究中高达 90% 的最大可实现 F1 分数可以通过重用工业案例中的训练模型来实现，并且仅使用低至 5 个标记样本对其进行微调，这将手动标记工作量减少了 98%。
Many software engineering tasks, such as testing, debugging, and anomaly detection can benefit from the ability to infer a behavioral model of the software. Most existing inference approaches assume access to code to collect execution sequences. In this paper, we investigate a black-box scenario, where the system under analysis cannot be instrumented in this fashion. This scenario is particularly common when it comes to the analysis of control system logs, which often take the form of continuous signals. In this situation, an execution trace amounts to a multivariate time-series of input and output signals, where different states of the system correspond to different “phases” in the time-series. From an inference perspective, the challenge is to detect when these phase changes take place. Unfortunately, most existing solutions are either univariate, make assumptions about the data distribution, or have limited learning power. In this paper we propose a hybrid deep neural network that accepts as input a multivariate time series and applies a set of convolutional and recurrent layers to learn the non-linear correlations between signals and the patterns over time. We show how this approach can be used to accurately detect state changes, and how the inferred models can be successfully applied to transfer-learning scenarios, to accurately process traces from different products with similar execution characteristics. Our experimental results on two UAV autopilot case studies (one industrial and one open-source) indicate that our approach is highly accurate (over 90% F1 score for state classification) and significantly improves baselines (by up to 102% for change point detection). Using transfer learning we also show that up to 90% of the maximum achievable F1 scores in the open-source case study can be achieved by reusing the trained models from the industrial case and only fine tuning them using as low as 5 labeled samples, which reduces the manual labeling effort by 98%.