Enhancing Model-based Testing using Software Analytics

软件正在被整合到越来越多的任务和安全关键系统中，包括实时嵌入式系统(例如巡航控制和空中交通管制系统)、通信平台(例如BlackBerry和Rogers无线网络)、电子银行基础设施(例如Interac和Visa)、电子商务系统(例如eBay拍卖系统和Amazon Elastic Compute Cloud)以及未来的电子医疗网络(例如加拿大的Health InfoStructure)。这些系统为我们的社会提供中心和关键的服务，因此需要高质量的软件。然而，历史上充斥着软件故障，导致了无数的问题，例如：飞机坠毁，患者死于错误的药物，关键的金融系统崩溃。我的研究目标是提高软件的质量。 我的项目侧重于将软件测试作为软件行业中最常用的质量保证方法。最近的研究表明，系统的测试自动化潜在地提高了测试的有效性，并降低了测试成本。因此，我的目标是研究最多的系统测试自动化技术之一，基于模型的测试(Model-Based Testing，MBT)，它从系统的规范模型自动生成测试。我的计划的目标是增强MBT，使其成为行业中常用的测试自动化技术。 有许多障碍使这一目标成为一个挑战，包括行业中缺乏规范模型(MBT的输入)及其可伸缩性和成本效益，特别是在超大规模系统、开源系统和敏捷实践中。我已经在解决其中一些关切方面取得了进展。通过我的博士研究，我专注于MBT在嵌入式系统中的使用，并提高了它在大型工业系统中的可扩展性。 在这个项目中，我计划通过关注其他问题来进一步增强MBT，例如缺乏规范模型和MBT在不同环境中的成本效益。总体思路是使用软件分析领域的一些最新和有前途的技术。这些技术将分析关于源代码控制系统、缺陷跟踪系统、需求文档、性能数据集、执行日志、邮件列表，甚至社交媒体的大量可用数据，以向MBT测试人员提供建议。 我的目标是开发一种半自动的模型生成技术，模型推荐系统，它将帮助测试人员构建规范模型，降低构建成本，并潜在地增加行业中规范模型的可用性。将开发的另外两种增强MBT的技术是具有风险意识的MBT和自适应MBT。有风险意识的MBT将把测试资源分配给风险更高的使用场景，在这些场景中可能会出现更大的问题；而自适应MBT将随着时间的推移和软件系统的发展控制MBT的质量。 建议的技术将在大型软件系统上进行经验评估，并与现有的软件测试自动化技术进行比较。这项研究将促进软件测试的研究和实践，并通过提供一种手段来提高软件行业的产品和服务质量，从而影响软件行业。由于软件测试是几乎所有软件开发过程中最常用的技术之一，加拿大许多行业的软件社区，如IT、金融、国防、石油和天然气等，将处于有利地位，通过降低人员成本和提高软件质量来增强国际竞争力，从而受益于这项研究的结果。

Software is being incorporated into an ever-increasing number of mission and safety-critical systems, including real-time embedded systems (e.g., cruise controls and air traffic control systems), communication platforms (e.g., the BlackBerry and Rogers wireless networks), e-banking infrastructures (e.g., Interac and Visa), e-commerce systems (e.g., the eBay auction system and the Amazon Elastic Compute Cloud), and future e-healthcare networks (e.g., Canada’s Health Infostructure). Such systems provide central and crucial services to our society and thus require high quality software. However, history is full of software failures that have caused numerous problems such as: aircraft have crashed, patients have died from incorrect medication, and key financial systems have broken down. My research is directed towards improving the quality of software. My program focuses on software testing as the most commonly used method of quality assurance in the software industry. Recent research has shown that systematic test automation potentially increases the effectiveness of testing and reduces its cost. Therefore, I will target one of the most studied systematic test automation techniques, model-based testing (MBT), which automatically generates tests from specification models of the system. The goal of my program is to enhance MBT so that it becomes the customary test automation technique in the industry. There are a number of hurdles that make this goal a challenge including the lack of specification models (inputs of MBT) in industry and its scalability and cost-effectiveness, particularly in ultra-large-scale systems, open source systems, and agile practices. I have already made progress in addressing a number of these concerns. Through my Ph.D. research, I focused on the use of MBT in embedded systems and improved its scalability in large industrial systems. In this program, I plan to further enhance MBT by focusing on the other concerns such as the lack of specification models and the cost-effectiveness of MBT in different contexts. The general idea is to use some recent and promising techniques from the field of Software Analytics. These techniques will analyze the large amount of data available about source control systems, defect tracking systems, requirement documents, performance datasets, execution logs, mailing lists, and even social media to provide recommendations to the MBT testers. I aim to develop a semi-automated model generation technique, Model Recommendation System, that will help the testers build specification models, reducing the build costs and potentially increasing the availability of specification models in the industry. Two other techniques that will be developed to enhance MBT are a Risk-aware MBT and an Adaptive MBT. Risk-aware MBT will allocate testing resources to the more risky usage scenarios, which are where the greater problems can be expected; and Adaptive MBT will control the quality of MBT over time and during the evolutions of the software systems. The proposed techniques will be empirically evaluated on large software systems and compared against existing software test automation techniques. This research will enhance both research and practice of software testing and impact the software industry by providing a means to improve the quality of their products and services. Since software testing is one of the most commonly used techniques in almost every software development processes, Canadian software communities in many sectors, such as IT, finance, defense, oil and gas, etc., will be well positioned to benefit from the outcomes of this research, through decreased personnel costs and increased software quality leading to enhanced competitiveness internationally.