NeuRI: Diversifying DNN Generation via Inductive Rule Inference
NeuRI: Diversifying DNN Generation via Inductive Rule Inference
复制标题
DOI：
10.1145/3611643.3616337
复制
发表时间：
2023-02
期刊：
影响因子：
--
通讯作者：
Jiawei Liu;Jinjun Peng;Yuyao Wang;Lingming Zhang
中科院分区：
文献类型：
--
作者：
Jiawei Liu;Jinjun Peng;Yuyao Wang;Lingming Zhang
深度学习（DL）在各个行业中被广泛用于改善决策和自动化流程，由不断发展的DL库和编译器驱动。DL系统的正确性对于DL应用程序的信任至关重要。因此，最近的研究浪潮一直在研究测试用例的自动合成（即，DNN模型和它们的输入）用于模糊DL系统。然而，现有的模型生成器只继承了有限数量的运营商，缺乏能力，无处不在的模型运营商的约束。为了应对这一挑战，我们提出了NeuRI，一个完全自动化的方法来生成有效的和多样化的DL模型组成的数百种类型的运营商。NeuRI采用了三个步骤的过程：（i）从各种来源收集有效和无效的API痕迹;（ii）应用归纳程序合成的痕迹，以推断的约束条件，构建有效的模型;和（iii）使用混合模型生成，其中包括符号和具体的运营商。我们的评估表明，NeuRI将TensorFlow和PyTorch的分支覆盖率提高了24%和15%，超过了最先进的模型级模糊器。NeuRI在四个月内发现了PyTorch和TensorFlow的100个新bug，其中81个已经修复或确认。其中，9个bug被标记为高优先级或安全漏洞，占该期间所有高优先级bug的10%。开源开发人员认为我们报告的错误诱导测试是“高质量”和“实践中常见的”。
Deep Learning (DL) is prevalently used in various industries to improve decision-making and automate processes, driven by the ever-evolving DL libraries and compilers. The correctness of DL systems is crucial for trust in DL applications. As such, the recent wave of research has been studying the automated synthesis of test-cases (i.e., DNN models and their inputs) for fuzzing DL systems. However, existing model generators only subsume a limited number of operators, lacking the ability to pervasively model operator constraints. To address this challenge, we propose NeuRI, a fully automated approach for generating valid and diverse DL models composed of hundreds of types of operators. NeuRI adopts a three-step process: (i) collecting valid and invalid API traces from various sources; (ii) applying inductive program synthesis over the traces to infer the constraints for constructing valid models; and (iii) using hybrid model generation which incorporates both symbolic and concrete operators. Our evaluation shows that NeuRI improves branch coverage of TensorFlow and PyTorch by 24% and 15% over the state-of-the-art model-level fuzzers. NeuRI finds 100 new bugs for PyTorch and TensorFlow in four months, with 81 already fixed or confirmed. Of these, 9 bugs are labelled as high priority or security vulnerability, constituting 10% of all high-priority bugs of the period. Open-source developers regard error-inducing tests reported by us as "high-quality" and "common in practice".