Input Distribution Coverage: Measuring Feature Interaction Adequacy in Neural Network Testing
Input Distribution Coverage: Measuring Feature Interaction Adequacy in Neural Network Testing
复制标题
DOI：
10.1145/3576040
复制
发表时间：
2022-12
影响因子：
4.4
通讯作者：
Swaroopa Dola;Matthew B. Dwyer;M. Soffa
中科院分区：
文献类型：
--
作者：
Swaroopa Dola;Matthew B. Dwyer;M. Soffa
近年来，由于深度神经网络（dnn）在许多应用中的应用，其测试引起了人们的极大兴趣。黑盒测试充分性度量对于指导覆盖输入域的测试过程是有用的。然而，缺乏输入规范使得在DNN测试中应用黑盒测试充分性度量具有挑战性。输入分布覆盖（IDC）框架通过使用变分自编码器来学习输入分布的低维潜在表示，然后使用该潜在空间作为覆盖域进行测试，从而解决了这一挑战。IDC在潜在空间的划分上应用组合交互测试来衡量测试的充分性。经验评估表明，IDC具有成本效益，能够检测测试输入中的特征多样性，并且对使用不同DNN测试生成方法生成的测试输入比以前的工作更敏感。研究结果表明，IDC克服了白盒深度神经网络覆盖方法的几个局限性，它从不现实的输入中剔除了覆盖范围，并能够计算测试充分性指标，从而捕获深度神经网络输入空间中存在的特征多样性。
Testing deep neural networks (DNNs) has garnered great interest in the recent years due to their use in many applications. Black-box test adequacy measures are useful for guiding the testing process in covering the input domain. However, the absence of input specifications makes it challenging to apply black-box test adequacy measures in DNN testing. The Input Distribution Coverage (IDC) framework addresses this challenge by using a variational autoencoder to learn a low dimensional latent representation of the input distribution, and then using that latent space as a coverage domain for testing. IDC applies combinatorial interaction testing on a partitioning of the latent space to measure test adequacy. Empirical evaluation demonstrates that IDC is cost-effective, capable of detecting feature diversity in test inputs, and more sensitive than prior work to test inputs generated using different DNN test generation methods. The findings demonstrate that IDC overcomes several limitations of white-box DNN coverage approaches by discounting coverage from unrealistic inputs and enabling the calculation of test adequacy metrics that capture the feature diversity present in the input space of DNNs.