Enhancing Model-based Testing using Software Analytics

软件正在被纳入越来越多的任务和安全关键系统，包括实时嵌入式系统（例如巡航控制和空中交通管制系统）、通信平台（例如黑莓和罗杰斯无线网络）、电子银行基础设施（例如Interac和Visa）、电子商务系统（例如eBay拍卖系统和亚马逊弹性计算云）和未来的电子医疗保健网络（例如加拿大的卫生信息结构）。这些系统为我们的社会提供了核心和关键的服务，因此需要高质量的软件。然而，历史上充满了软件故障，这些故障导致了许多问题，例如：飞机坠毁，病人死于错误的药物治疗，以及关键的金融系统崩溃。我的研究方向是提高软件的质量。*我的课程侧重于软件测试，这是软件行业中最常用的质量保证方法。最近的研究表明，系统的测试自动化潜在地提高了测试的有效性，并降低了测试的成本。因此，我将针对研究最多的系统测试自动化技术之一，基于模型的测试（MBT），它自动从系统的规范模型生成测试。我的程序的目标是增强MBT，使其成为行业中常用的测试自动化技术。*有许多障碍使这一目标成为挑战，包括工业中缺乏规范模型（MBT的输入）及其可伸缩性和成本效益，特别是在超大规模系统、开源系统和敏捷实践中。我已经在解决其中一些问题方面取得了进展。在我的博士研究中，我专注于MBT在嵌入式系统中的使用，并提高了它在大型工业系统中的可扩展性。*在这个项目中，我计划进一步加强MBT，重点关注其他问题，如缺乏规范模型和不同情况下MBT的成本效益。一般的想法是使用软件分析领域的一些最新和有前途的技术。这些技术将分析大量关于源代码控制系统、缺陷跟踪系统、需求文档、性能数据集、执行日志、邮件列表，甚至社会媒体的可用数据，从而向MBT测试人员提供建议。*我的目标是开发一种半自动化的模型生成技术，模型推荐系统，它将帮助测试人员构建规范模型，降低构建成本，并潜在地增加行业中规范模型的可用性。将开发的另外两种增强主战坦克的技术是风险感知主战坦克和自适应主战坦克。有风险意识的MBT将把测试资源分配给风险更大的使用场景，在这些场景中可能会出现更大的问题；自适应MBT将随着时间的推移和软件系统的演变控制MBT的质量。*建议的技术将在大型软件系统上进行经验评估，并与现有的软件测试自动化技术进行比较。这项研究将加强软件测试的研究和实践，并通过提供一种提高产品和服务质量的方法来影响软件行业。由于软件测试是几乎每个软件开发过程中最常用的技术之一，加拿大软件社区在许多部门，如IT、金融、国防、石油和天然气等，将很好地从这项研究的结果中受益，通过降低人员成本和提高软件质量来增强国际竞争力。

Software is being incorporated into an ever-increasing number of mission and safety-critical systems, including real-time embedded systems (e.g., cruise controls and air traffic control systems), communication platforms (e.g., the BlackBerry and Rogers wireless networks), e-banking infrastructures (e.g., Interac and Visa), e-commerce systems (e.g., the eBay auction system and the Amazon Elastic Compute Cloud), and future e-healthcare networks (e.g., Canada's Health Infostructure). Such systems provide central and crucial services to our society and thus require high quality software. However, history is full of software failures that have caused numerous problems such as: aircraft have crashed, patients have died from incorrect medication, and key financial systems have broken down. My research is directed towards improving the quality of software. *My program focuses on software testing as the most commonly used method of quality assurance in the software industry. Recent research has shown that systematic test automation potentially increases the effectiveness of testing and reduces its cost. Therefore, I will target one of the most studied systematic test automation techniques, model-based testing (MBT), which automatically generates tests from specification models of the system. The goal of my program is to enhance MBT so that it becomes the customary test automation technique in the industry. *There are a number of hurdles that make this goal a challenge including the lack of specification models (inputs of MBT) in industry and its scalability and cost-effectiveness, particularly in ultra-large-scale systems, open source systems, and agile practices. I have already made progress in addressing a number of these concerns. Through my Ph.D. research, I focused on the use of MBT in embedded systems and improved its scalability in large industrial systems. *In this program, I plan to further enhance MBT by focusing on the other concerns such as the lack of specification models and the cost-effectiveness of MBT in different contexts. The general idea is to use some recent and promising techniques from the field of Software Analytics. These techniques will analyze the large amount of data available about source control systems, defect tracking systems, requirement documents, performance datasets, execution logs, mailing lists, and even social media to provide recommendations to the MBT testers. *I aim to develop a semi-automated model generation technique, Model Recommendation System, that will help the testers build specification models, reducing the build costs and potentially increasing the availability of specification models in the industry. Two other techniques that will be developed to enhance MBT are a Risk-aware MBT and an Adaptive MBT. Risk-aware MBT will allocate testing resources to the more risky usage scenarios, which are where the greater problems can be expected; and Adaptive MBT will control the quality of MBT over time and during the evolutions of the software systems. *The proposed techniques will be empirically evaluated on large software systems and compared against existing software test automation techniques. This research will enhance both research and practice of software testing and impact the software industry by providing a means to improve the quality of their products and services. Since software testing is one of the most commonly used techniques in almost every software development processes, Canadian software communities in many sectors, such as IT, finance, defense, oil and gas, etc., will be well positioned to benefit from the outcomes of this research, through decreased personnel costs and increased software quality leading to enhanced competitiveness internationally.