The Automation of (Black-box) Software Testing

目前的软件测试方法无法满足世界经济的目标。与未发现的缺陷相关的问题和问题现在估计超过全球GDP的0.6%。在加拿大，这相当于90亿（美国）美元;全球约4000亿美元。因此，本研究计划旨在建立激进的新方法来进行软件测试，通过降低测试成本，同时增加收益（发现缺陷的数量）来减轻这种负担。该研究的一个基本原则是，软件测试需要是一个高度自动化的过程，测试技术需要将被测软件（SUT）视为一个“黑盒”;这使得黑盒技术可以应用于每一个软件而无需修改。由于现代计算向多核和云计算平台的转变，这种方法变得非常可行。这些平台允许将大量的计算资源用于测试目标，使自动化系统能够分析大量的测试场景。据推测，更有效的测试形式可以通过将测试视为一个相对独立的SUT过程来实现。该方法通过大量的测试场景（意味着自动化是必不可少的）轰炸SUT，产生了上级结果（成本更低，发现更多的缺陷）。这个雄心勃勃的研究计划将解决有效实现测试用例生成策略，程序执行建模技术和测试Oracle定义的基本问题，以实现这一愿景。该提案将产生大量重要研究成果;具体来说，根据历史趋势，博士该小组的学生大约每年都会出版一份高质量的期刊。因此，预计所申请的资金将直接产生16篇高质量的期刊论文，这些论文将在IEEE和ACM Transaction等领先场所发表。该赠款还将产生大量的HQP（4个直接与发现赠款资金，整个提案预计将产生10个HQP），所有在自动化软件测试的新兴领域的高素质。希望这些人能够成为一个枢纽，推动加拿大走在这一新的重要经济活动的前列。最后，预计在项目期间开发的软件测试基础设施可以成为软件即服务产品，从而能够成立一个向全世界整个软件界提供服务的新企业。这样的组织非常有吸引力，因为（1）通过利用预先存在的云基础设施作为服务，它可以以最低的资本要求建立;（2）没有任何物理足迹，公司在创建时就成为一个全球性实体。**

Current approaches to Software Testing fail to meet the objectives of the world economy. Problems and issues associated with undiscovered defects are now estimated to exceed more than 0.6% of global GDP. In Canadian terms, this equates to 9 Billion (U.S.) dollars; and approximately 400 Billion (U.S) dollars worldwide. Hence, this research program seeks to establish radical new approaches to software testing to contribute to alleviating this burden by reducing the costs of testing while increasing the benefits (the number of discovered defects). A basic tenet of the research is that software testing needs to be a highly automated process and that testing techniques need to treat the software under test (SUT) as a “black-box”; this allows black-box techniques to be applied to every piece of software without modification. This approach has become highly feasible because of the shift in modern computing to multi-core and cloud computing platforms. These platforms allow massive amounts of computing resources to be targeted towards testing objectives enabling automated systems to analyse vast numbers of testing scenarios. It is hypothesized that more efficient forms of testing can be achieved by viewing testing as a relatively SUT-independent process. The approach produces superior results (costs less, finds more defects) by bombarding the SUT with vast volumes of test scenarios (implying automation is essential). This ambitious program of research will solve fundamental problems in the efficient realization of test case generation strategies, program execution modelling techniques and test oracle definitions to realize this vision.******The proposal will generate a significant volume of important research results; specifically, based upon historical trends, Ph.D. students within the group generate a high-quality journal publication approximately every year. Hence, it is expected that the requested funding will directly generate 16 high-quality journals papers to be published in leading venues such as IEEE and ACM Transaction venues. The grant will also produce a large number of HQP (4 directly with the Discovery grant funding, with the entire proposal expected to produce 10 HQP), all highly qualified in the burgeoning area of automated software testing. It is hoped that these individuals can become a hub and propel Canada to the forefront of this new significant economic activity. Finally, it is expected that the software testing infrastructure developed during the project can become a software-as-a-service product, enabling the formation of a new enterprise providing services to the entire software community worldwide. Such an organization is highly attractive because (1) by utilizing pre-existing cloud-infrastructures as a service, it can be established with minimal capital requirements; and (2) without any physical footprint, the company becomes a worldwide entity upon creation. **