Distribution Models for Falsification and Verification of DNNs
Distribution Models for Falsification and Verification of DNNs
复制标题
DOI：
10.1109/ase51524.2021.9678590
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Felipe R. Toledo;David Shriver;Sebastian G. Elbaum;Matthew B. Dwyer
中科院分区：
文献类型：
--
作者：
Felipe R. Toledo;David Shriver;Sebastian G. Elbaum;Matthew B. Dwyer
与输入分布无关的DNN验证和验证方法浪费了不相关的输入，并报告了错误的属性违规。借鉴传统系统基于模型的验证和验证的大量工作，我们介绍了第一种利用环境模型将DNN伪造和验证集中在相关输入空间的方法。我们的方法DFV使用无监督学习自动构建输入分布模型，将该模型添加到DNN以强制所有输入都来自学习的分布，并将属性重新定义为分布模型的输入空间。这个转换的验证问题允许现有的DNN伪造和验证工具以输入分布为目标-避免考虑不可行的输入。我们对DFV的研究使用了7种伪造和验证工具，两种DNN定义在不同的数据集上，以及93种不同的分布模型，提供了明确的证据，证明工具发现的反例更能代表数据分布，并显示了DFV的性能如何在不同的领域，模型和工具之间变化。
DNN validation and verification approaches that are input distribution agnostic waste effort on irrelevant inputs and report false property violations. Drawing on the large body of work on model-based validation and verification of traditional systems, we introduce the first approach that leverages environmental models to focus DNN falsification and verification on the relevant input space. Our approach, DFV, automatically builds an input distribution model using unsupervised learning, prefixes that model to the DNN to force all inputs to come from the learned distribution, and reformulates the property to the input space of the distribution model. This transformed verification problem allows existing DNN falsification and verification tools to target the input distribution – avoiding consideration of infeasible inputs. Our study of DFV with 7 falsification and verification tools, two DNNs defined over different data sets, and 93 distinct distribution models, provides clear evidence that the counterexamples found by the tools are much more representative of the data distribution, and it shows how the performance of DFV varies across domains, models, and tools.