Mining Software Repositories and Information Visualization for Empirically Robust Testing of Variable Software

在现代软件系统中，“一刀切”已经不成立了，因为它们必须被定制以适应多样化和大量用户的需求。可变软件指的是这种类型的软件系统，它可以以多种方式进行配置，并且主要源于采用基于生成器的技术、成熟的软件产品线（SPL）方法、高级模块化范例、高度可配置的系统，或者特别的重用实践，统称为克隆和拥有。测试可变软件特别具有挑战性，因为必须测试的配置通常是大量的，这使得测试每个单独的配置变得不可行。组合交互测试（CIT）已经被提倡作为解决这一挑战的范例，因为可用的各种算法集可以用来计算用于测试的具有代表性的配置组。然而，这个研究项目的目标是解决几个悬而未决的问题。首先，对于变量软件的公开可用的案例研究，缺乏经验地评估和验证CIT技术。其次，迫切需要对应用于可变软件的不同CIT技术进行彻底的经验和比较评估。第三，没有足够的工具支持测试可变软件。当前的实践是插件和独立工具的不协调和不完整分类的特别集合。本研究计划的目标是：1)利用挖掘软件存储库的技术，收集可变软件系统的公开可用案例研究，对应用于可变软件的CIT技术进行彻底的实证评估。2)对应用于可变软件的CIT技术进行系统评估。3)利用信息可视化技术来更好地传达存在于可变软件案例研究中的大量信息。4)提供强大的工具支持，允许软件工程师设计、实现和执行可变软件的测试。第一个目标将通过研究诸如GitHub等存储库中可用的开源项目以及开发挖掘配置和故障数据的工具来实现。对于第二个目标，我们将系统地收集和编目用于可变软件的CIT测试的工具，并使用第一个目标的案例研究数据对它们进行评估。对于第三个目标，我们将探索和评估不同的可视化技术，以传达所有不同类型的信息，这些信息必须在测试可变软件时做出适当的工程决策。对于第四个目标，我们将开发一个框架，允许灵活地添加测试工具和可视化界面，为软件工程师提供测试任务的强大工具支持。

In modern software systems "one-size fits all" does not hold because they must be customized to fit the needs of diverse and large populations of users. Variable software refers to this type of software systems that can be configured in a large number of ways, and that stem primarily from the adoption of generator-based techniques, full-blown Software Product Line (SPL) approaches, advanced modularization paradigms, highly-configurable systems, or ad hoc reuse practices collectively called clone-and-own. Testing variable software is specially challenging because of the typically large number of configurations that must be tested, a fact that makes it unfeasible to test each individual configuration. Combinatorial Interaction Testing (CIT) has been advocated as a paradigm to address this challenge because of the diverse set of algorithms available which can be exploited to compute representative groups of configurations for testing. There are however several open problems that this research program aims to tackle. First, there is a critical lack of publicly-available case studies of variable software on which to empirically evaluate and validate CIT techniques. Second, there is a stark need of a thorough empirical and comparative evaluation of different CIT techniques that have been applied to variable software. Third, there is no adequate tool support for testing of variable software. The current practice is an ad-hoc collection of uncoordinated and incomplete assortment of plug-ins and stand-alone tools.The objectives for this research program are:1) Exploit techniques for mining software repositories to gather publicly-available case studies of variable software systems for a thorough empirical evaluation of CIT techniques applied to variable software. 2) Perform a systematic assessment of CIT techniques applied to variable software. 3) Leverage information visualization techniques to better convey the large amount of information present in variable software case studies. 4) Provide robust tool support that allows software engineers to design, implement, and carry out test of variable software.The first objective will be achieved by studying open source projects available in repositories such as GitHub and developing tools that mine configuration and fault data. For the second objective we will systematically collect and catalog tools for CIT testing of variable software, and evaluate them with the case studies' data of the first objective. For the third objective we will explore and evaluate different visualization techniques to convey all the distinct types of information that must be considered for making adequate engineering decisions when testing variable software. For the fourth objective, we will develop a framework that allows flexible addition of testing tools and visualization interfaces to provide software engineers with robust tool support of testing tasks.