DocTer: documentation-guided fuzzing for testing deep learning API functions
DocTer: documentation-guided fuzzing for testing deep learning API functions
复制标题
DOI：
10.1145/3533767.3534220
复制
发表时间：
2021-09
期刊：
影响因子：
--
通讯作者：
Danning Xie;Yitong Li;Mijung Kim;H. Pham;Lin Tan;X. Zhang;Michael W. Godfrey
中科院分区：
文献类型：
--
作者：
Danning Xie;Yitong Li;Mijung Kim;H. Pham;Lin Tan;X. Zhang;Michael W. Godfrey
输入约束对于许多软件开发任务都很有用。例如，函数的输入约束使得能够生成有效输入，即，输入遵循这些约束，以更深入地测试函数。深度学习（DL）库的API函数具有DL特定的输入约束，这些约束在自由格式的API文档中进行了非正式描述。现有的约束提取技术是无效的提取DL特定的输入约束。为了填补这一空白，我们设计并实现了一种新的技术-DocTer-分析API文档，以提取DL特定的输入约束DL API功能。DocTer提供了一种新颖的算法，该算法可以自动构建规则，以API描述的依赖解析树的形式从语法模式中提取API参数约束。然后将这些规则应用于流行DL库中的大量API文档，以提取其输入参数约束。为了证明提取的约束的有效性，DocTer使用这些约束来自动生成有效和无效的输入，以测试DL API函数。我们对三个流行的DL库（TensorFlow，PyTorch和MXNet）的评估显示，DocTer在提取输入约束方面的准确率为85.4%。DocTer从174个API函数中检测到94个错误，包括一个以前未知的安全漏洞，现在记录在CVE数据库中，而没有输入约束的基线技术只检测到59个错误。94个bug中的大多数（63个）是以前未知的，其中54个在我们报告后已经被开发人员修复或确认。此外，DocTer还检测到文档中的43个不一致之处，其中39个已经修复或确认。
Input constraints are useful for many software development tasks. For example, input constraints of a function enable the generation of valid inputs, i.e., inputs that follow these constraints, to test the function deeper. API functions of deep learning (DL) libraries have DL-specific input constraints, which are described informally in the free-form API documentation. Existing constraint-extraction techniques are ineffective for extracting DL-specific input constraints. To fill this gap, we design and implement a new technique—DocTer—to analyze API documentation to extract DL-specific input constraints for DL API functions. DocTer features a novel algorithm that automatically constructs rules to extract API parameter constraints from syntactic patterns in the form of dependency parse trees of API descriptions. These rules are then applied to a large volume of API documents in popular DL libraries to extract their input parameter constraints. To demonstrate the effectiveness of the extracted constraints, DocTer uses the constraints to enable the automatic generation of valid and invalid inputs to test DL API functions. Our evaluation on three popular DL libraries (TensorFlow, PyTorch, and MXNet) shows that DocTer’s precision in extracting input constraints is 85.4%. DocTer detects 94 bugs from 174 API functions, including one previously unknown security vulnerability that is now documented in the CVE database, while a baseline technique without input constraints detects only 59 bugs. Most (63) of the 94 bugs are previously unknown, 54 of which have been fixed or confirmed by developers after we report them. In addition, DocTer detects 43 inconsistencies in documents, 39 of which are fixed or confirmed.