Category partition black-box software testing: theory, tool support and experiments

类别划分是一种基于等价类划分和边界值分析的黑盒软件测试技术。它适用于许多情况：例如，C单元测试，Java单元/集成测试，用例系统测试。它还与其他技术相结合，例如扩展有限状态机的基于状态的测试。 CP首先确定被测功能的参数和环境变量。环境变量是被测程序执行环境中可能影响其行为的因素。然后将这些参数和环境变量设想为类别，这些类别是从测试的角度来看被认为是重要的特性。每个特征都导致了选择的定义（即，等价类）分割由特征（隐式）定义的值的域。然后，约束可以指定两个不同类别中的某些选项应该始终一起使用，永远不能一起使用，或者只能在特定条件下一起使用。然后，根据选择标准将选择组合以形成测试框架，同时满足约束。一个简单的充分性准则要求每个选择在测试框架集中至少出现一次。然后，根据选择规范，将每个测试帧与实际输入值（参数和环境变量）相关联，以生成测试用例。 当使用CP时，需要做出许多可能影响结果的决定（例如，成本、故障检测）。显然，选择的选择有影响：例如，定义不好的类别和选择将导致在测试活动中遗漏的错误;选择适当性标准很重要，因为标准并不都是同等要求的（即，成本）或有效（即，故障检测）;在考虑约束的同时，产生一组足以满足标准的测试框架的技术将产生影响;识别测试框架输入值的程序可能产生影响。 不幸的是，很少有人知道这些决定对应用CP结果的实际精确影响。 这个项目的目的就是填补这个空白。将创建工具支持，以促进在不同案例研究中使用CP和替代决定。将使用真实的和合成故障来评估许多备选方案对测试技术在查找故障时的有效性的影响。 研究结果将引起研究界的高度兴趣：这是第一次研究CP的许多替代应用。结果也将与行业高度相关：工具支持，实验证据导致更好的决策。结果将是非常有趣的教学目的：剪裁教学的技术，工具的可用性，包括在线版本。

Category partition (CP) is a black-box software testing technique based on equivalence class partitioning and boundary value analysis. It applies to many contexts: e.g., C unit testing, Java unit/integration testing, system testing of use cases. It has also been combined with other techniques such as state-based testing from extended finite state machines. CP begins by identifying the parameters and environment variables of a functionality under test. Environment variables are factors in the environment of execution of the program under test that may impact its behaviour. These parameters and environment variables are then envisioned into categories, which are characteristics that are deemed important from a testing viewpoint. Each characteristic leads to the definition of choices (i.e., equivalent classes) splitting the domain of values (implicitly) defined by the characteristic. Constraints can then specify that some choices from two different categories should always be used together, can never be used together, or can only be used together under certain condition. The choices are then combined to form test frames according to a selection criterion, while satisfying constraints. A simple adequacy criterion requires that each choice appears at least once in the set of test frames. Each test frame is then associated with actual input values (for parameters and environment variables), according to the choice specifications, to produce test cases. When using CP, one needs to take many decisions that can impact the result (e.g., cost, fault detection) of the testing campaign. Obviously the selection of choices has an impact: e.g., poorly defined categories and choices will lead to faults that slip through the testing activity; Choosing an adequacy criterion matters because criteria are not all equally demanding (i.e., cost) or effective (i.e., fault detection); The technology to produce a set of test frames adequate for a criterion, while accounting for constraints, will have an impact; The procedure to identify input values for test frames may have an impact. Unfortunately, very little is known about the actual, precise impact of those decisions on the result of applying CP. The purpose of this project is to fill this gap. Tool support will be created to facilitate the use of CP and alternative decisions, on different case studies. Real and synthetic faults will be used to evaluate the impact of the many alternatives on the effectiveness of the testing technique at finding faults. The results will be of high interest to the research community: it is the first time the many alternative applications of CP will be studied. Results will also be highly relevant to industry: tool support, experimental evidence leading to better decision making. Results will be highly interesting for teaching purposes: tailoring the teaching of the technique, availability of the tool, including an online version.