The Automation of (Black-box) Software Testing

点击翻译按钮获取中文摘要

Current approaches to Software Testing fail to meet the objectives of the world economy. Problems and issues associated with undiscovered defects are now estimated to exceed more than 0.6% of global GDP. In Canadian terms, this equates to 9 Billion (U.S.) dollars; and approximately 400 Billion (U.S) dollars worldwide. Hence, this research program seeks to establish radical new approaches to software testing to contribute to alleviating this burden by reducing the costs of testing while increasing the benefits (the number of discovered defects). A basic tenet of the research is that software testing needs to be a highly automated process and that testing techniques need to treat the software under test (SUT) as a “black-box”; this allows black-box techniques to be applied to every piece of software without modification. This approach has become highly feasible because of the shift in modern computing to multi-core and cloud computing platforms. These platforms allow massive amounts of computing resources to be targeted towards testing objectives enabling automated systems to analyse vast numbers of testing scenarios. It is hypothesized that more efficient forms of testing can be achieved by viewing testing as a relatively SUT-independent process. The approach produces superior results (costs less, finds more defects) by bombarding the SUT with vast volumes of test scenarios (implying automation is essential). This ambitious program of research will solve fundamental problems in the efficient realization of test case generation strategies, program execution modelling techniques and test oracle definitions to realize this vision.******The proposal will generate a significant volume of important research results; specifically, based upon historical trends, Ph.D. students within the group generate a high-quality journal publication approximately every year. Hence, it is expected that the requested funding will directly generate 16 high-quality journals papers to be published in leading venues such as IEEE and ACM Transaction venues. The grant will also produce a large number of HQP (4 directly with the Discovery grant funding, with the entire proposal expected to produce 10 HQP), all highly qualified in the burgeoning area of automated software testing. It is hoped that these individuals can become a hub and propel Canada to the forefront of this new significant economic activity. Finally, it is expected that the software testing infrastructure developed during the project can become a software-as-a-service product, enabling the formation of a new enterprise providing services to the entire software community worldwide. Such an organization is highly attractive because (1) by utilizing pre-existing cloud-infrastructures as a service, it can be established with minimal capital requirements; and (2) without any physical footprint, the company becomes a worldwide entity upon creation. **