Enhancing Model-based Testing using Software Analytics

点击翻译按钮获取中文摘要

Software is being incorporated into an ever-increasing number of mission and safety-critical systems, including real-time embedded systems (e.g., cruise controls and air traffic control systems), communication platforms (e.g., the BlackBerry and Rogers wireless networks), e-banking infrastructures (e.g., Interac and Visa), e-commerce systems (e.g., the eBay auction system and the Amazon Elastic Compute Cloud), and future e-healthcare networks (e.g., Canada’s Health Infostructure). Such systems provide central and crucial services to our society and thus require high quality software. However, history is full of software failures that have caused numerous problems such as: aircraft have crashed, patients have died from incorrect medication, and key financial systems have broken down. My research is directed towards improving the quality of software. My program focuses on software testing as the most commonly used method of quality assurance in the software industry. Recent research has shown that systematic test automation potentially increases the effectiveness of testing and reduces its cost. Therefore, I will target one of the most studied systematic test automation techniques, model-based testing (MBT), which automatically generates tests from specification models of the system. The goal of my program is to enhance MBT so that it becomes the customary test automation technique in the industry. There are a number of hurdles that make this goal a challenge including the lack of specification models (inputs of MBT) in industry and its scalability and cost-effectiveness, particularly in ultra-large-scale systems, open source systems, and agile practices. I have already made progress in addressing a number of these concerns. Through my Ph.D. research, I focused on the use of MBT in embedded systems and improved its scalability in large industrial systems. In this program, I plan to further enhance MBT by focusing on the other concerns such as the lack of specification models and the cost-effectiveness of MBT in different contexts. The general idea is to use some recent and promising techniques from the field of Software Analytics. These techniques will analyze the large amount of data available about source control systems, defect tracking systems, requirement documents, performance datasets, execution logs, mailing lists, and even social media to provide recommendations to the MBT testers. I aim to develop a semi-automated model generation technique, Model Recommendation System, that will help the testers build specification models, reducing the build costs and potentially increasing the availability of specification models in the industry. Two other techniques that will be developed to enhance MBT are a Risk-aware MBT and an Adaptive MBT. Risk-aware MBT will allocate testing resources to the more risky usage scenarios, which are where the greater problems can be expected; and Adaptive MBT will control the quality of MBT over time and during the evolutions of the software systems. The proposed techniques will be empirically evaluated on large software systems and compared against existing software test automation techniques. This research will enhance both research and practice of software testing and impact the software industry by providing a means to improve the quality of their products and services. Since software testing is one of the most commonly used techniques in almost every software development processes, Canadian software communities in many sectors, such as IT, finance, defense, oil and gas, etc., will be well positioned to benefit from the outcomes of this research, through decreased personnel costs and increased software quality leading to enhanced competitiveness internationally.