Automatically Generating Precise Oracles from Structured Natural Language Specifications
Automatically Generating Precise Oracles from Structured Natural Language Specifications
复制标题
DOI：
10.1109/icse.2019.00035
复制
发表时间：
2019-05
期刊：
影响因子：
--
通讯作者：
Manish Motwani;Yuriy Brun
中科院分区：
文献类型：
--
作者：
Manish Motwani;Yuriy Brun
软件规范通常使用自然语言来描述期望的行为，但是这样的规范很难自动验证。我们介绍了Swami，一种从结构化的自然语言规范中提取测试预言并生成可执行测试的自动化技术。Swami关注的是异常行为和边界条件，这些通常会导致字段失败，但开发人员经常无法手动编写测试。在官方JavaScript规范（ECMA-262）上进行评估后，Swami生成的98.4%的测试都精确地符合规范。使用Swami增强开发人员编写的测试套件提高了覆盖率，并确定了Rhino中1个以前未知的缺陷和15个缺失的JavaScript特性，Node.js中1个以前未知的缺陷，以及ECMA-262规范中的18个语义歧义。
Software specifications often use natural language to describe the desired behavior, but such specifications are difficult to verify automatically. We present Swami, an automated technique that extracts test oracles and generates executable tests from structured natural language specifications. Swami focuses on exceptional behavior and boundary conditions that often cause field failures but that developers often fail to manually write tests for. Evaluated on the official JavaScript specification (ECMA-262), 98.4% of the tests Swami generated were precise to the specification. Using Swami to augment developer-written test suites improved coverage and identified 1 previously unknown defect and 15 missing JavaScript features in Rhino, 1 previously unknown defect in Node.js, and 18 semantic ambiguities in the ECMA-262 specification.