Category partition black-box software testing: theory, tool support and experiments

类别划分是一种基于等价类划分和边值分析的黑盒软件测试技术。它适用于许多环境：例如，C单元测试、Java单元/集成测试、用例的系统测试。它还与其他技术相结合，例如来自扩展有限状态机的基于状态的测试。*CP从识别被测功能的参数和环境变量开始。环境变量是被测程序执行环境中可能影响其行为的因素。然后，这些参数和环境变量被设想成类别，从测试的角度来看，这些类别被认为是重要的特征。每个特征导致选择的定义(即，等价类)，分割由该特征定义的值域(隐含地)。然后，约束可以指定来自两个不同类别的某些选项应该始终一起使用，永远不能一起使用，或者只能在特定条件下一起使用。然后根据选择标准组合这些选择以形成测试帧，同时满足约束。一个简单的充分性标准要求每个选项在测试帧集中至少出现一次。然后，根据选择规范，将每个测试帧与实际输入值(参数和环境变量)相关联，以产生测试用例。*使用CP时，需要做出许多决策，这些决策可能会影响测试活动的结果(例如，成本、故障检测)。显然，选项的选择有影响：例如，定义不好的类别和选择将导致测试活动中漏掉的错误；选择充分性标准很重要，因为标准并不都是同样苛刻的(即成本)或有效的(即故障检测)；产生一组足以满足标准的测试框架的技术，同时考虑到约束，将产生影响；确定测试框输入值的程序可能会产生影响。*不幸的是，人们对这些决定对应用CP的结果的实际、准确的影响知之甚少。*本项目的目的是填补这一空白。将建立工具支持，以促进在不同的案例研究中使用国家方案和备选决策。真实故障和合成故障将被用来评估许多替代方案对测试技术在发现故障时的有效性的影响。*研究结果将引起研究界的高度兴趣：这将是第一次研究CP的许多替代应用。结果也将与行业高度相关：工具支持、有助于更好决策的实验证据。结果将是非常有趣的教学目的：量身定做的技术教学，工具的可用性，包括一个在线版本。

Category partition (CP) is a black-box software testing technique based on equivalence class partitioning and boundary value analysis. It applies to many contexts: e.g., C unit testing, Java unit/integration testing, system testing of use cases. It has also been combined with other techniques such as state-based testing from extended finite state machines.***CP begins by identifying the parameters and environment variables of a functionality under test. Environment variables are factors in the environment of execution of the program under test that may impact its behaviour. These parameters and environment variables are then envisioned into categories, which are characteristics that are deemed important from a testing viewpoint. Each characteristic leads to the definition of choices (i.e., equivalent classes) splitting the domain of values (implicitly) defined by the characteristic. Constraints can then specify that some choices from two different categories should always be used together, can never be used together, or can only be used together under certain condition. The choices are then combined to form test frames according to a selection criterion, while satisfying constraints. A simple adequacy criterion requires that each choice appears at least once in the set of test frames. Each test frame is then associated with actual input values (for parameters and environment variables), according to the choice specifications, to produce test cases. ***When using CP, one needs to take many decisions that can impact the result (e.g., cost, fault detection) of the testing campaign. Obviously the selection of choices has an impact: e.g., poorly defined categories and choices will lead to faults that slip through the testing activity; Choosing an adequacy criterion matters because criteria are not all equally demanding (i.e., cost) or effective (i.e., fault detection); The technology to produce a set of test frames adequate for a criterion, while accounting for constraints, will have an impact; The procedure to identify input values for test frames may have an impact.***Unfortunately, very little is known about the actual, precise impact of those decisions on the result of applying CP.***The purpose of this project is to fill this gap. Tool support will be created to facilitate the use of CP and alternative decisions, on different case studies. Real and synthetic faults will be used to evaluate the impact of the many alternatives on the effectiveness of the testing technique at finding faults.***The results will be of high interest to the research community: it is the first time the many alternative applications of CP will be studied. Results will also be highly relevant to industry: tool support, experimental evidence leading to better decision making. Results will be highly interesting for teaching purposes: tailoring the teaching of the technique, availability of the tool, including an online version.