The Automation of (Black-box) Software Testing

当前的软件测试方法不能满足世界经济的目标。与未发现的缺陷相关的问题和问题现在估计超过全球GDP的0.6%。在加拿大，这相当于90亿美元；全球约有4000亿美元。因此，该研究计划寻求建立根本性的软件测试新方法，以在增加收益（发现缺陷的数量）的同时减少测试的成本，从而减轻这种负担。该研究的一个基本原则是，软件测试需要是一个高度自动化的过程，测试技术需要将被测软件（SUT）视为一个“黑盒”；这使得黑盒技术无需修改就可以应用于软件的每个部分。由于现代计算向多核和云计算平台的转变，这种方法变得非常可行。这些平台允许大量的计算资源瞄准测试目标，使自动化系统能够分析大量的测试场景。假设通过将测试视为相对独立于sut的过程，可以实现更有效的测试形式。该方法通过大量的测试场景（意味着自动化是必要的）轰炸SUT，产生了更好的结果（成本更低，发现了更多的缺陷）。这个雄心勃勃的研究计划将解决有效实现测试用例生成策略、程序执行建模技术和测试oracle定义方面的基本问题，以实现这一愿景。

Current approaches to Software Testing fail to meet the objectives of the world economy. Problems and issues associated with undiscovered defects are now estimated to exceed more than 0.6% of global GDP. In Canadian terms, this equates to 9 Billion (U.S.) dollars; and approximately 400 Billion (U.S) dollars worldwide. Hence, this research program seeks to establish radical new approaches to software testing to contribute to alleviating this burden by reducing the costs of testing while increasing the benefits (the number of discovered defects). A basic tenet of the research is that software testing needs to be a highly automated process and that testing techniques need to treat the software under test (SUT) as a “black-box”; this allows black-box techniques to be applied to every piece of software without modification. This approach has become highly feasible because of the shift in modern computing to multi-core and cloud computing platforms. These platforms allow massive amounts of computing resources to be targeted towards testing objectives enabling automated systems to analyse vast numbers of testing scenarios. It is hypothesized that more efficient forms of testing can be achieved by viewing testing as a relatively SUT-independent process. The approach produces superior results (costs less, finds more defects) by bombarding the SUT with vast volumes of test scenarios (implying automation is essential). This ambitious program of research will solve fundamental problems in the efficient realization of test case generation strategies, program execution modelling techniques and test oracle definitions to realize this vision.