CPOT-SM – Complete Property-oriented Testing with Symbolic Methods

完整的测试方法为测试套件生成技术提供了在关于被测系统（SUT）的某些假设下保证故障覆盖的技术。虽然它们一直被认为是一个有趣的研究领域，但它们的实际适用性经常受到质疑，因为当希望在现实假设下实现完全故障覆盖时，要应用的测试套件大小难以管理。这种情况在过去几年中发生了变化，因为可以证明等效类划分技术可以应用于工业规模的sut。在这个项目中，我们计划在面向属性的测试（POT）的背景下利用这些最近的结果。与一般的基于模型的测试（MBT）相比，POT只关注单个SUT需求的彻底测试。因此，POT不需要描述SUT预期行为的完整模型，但可以基于(1)时间逻辑中声明的规范，(2)部分模型，或(3)抽象模型。因此，通过POT启用SUT验证的成本通常低于完整MBT方法的成本。缺点是，目前还没有针对属性测试完备性的综合理论。因此，这个项目的主要目标是详细阐述新的POT方法，允许生成具有保证故障覆盖率的测试套件。这将通过涉及模型和时间逻辑规范的混合方法来实现。为了处理模型的状态空间复杂性，将应用符号方法，其中具体的输入、输出和SUT状态被标识输入、输出和状态类的一阶公式所取代。作为一个额外的挑战，我们将在自治系统的背景下研究POT。由于POT与MBT有密切的联系，一个支持性的工作包将集中于具有无限输入、内部状态和输出域的系统的完整的基于模型的测试理论。进一步的工作包将研究生产线测试中可配置系统的问题，并将现有的等价类理论扩展到配置参数。CPOT-SM研究成果将作为实验原型在现有的工业强度MBT工具中实施。拟定的方法将在与国际研究伙伴进行的案例研究中在这一工具框架中得到验证。案例研究涵盖了航空电子、铁路、自动驾驶汽车和机器人等领域。

Complete testing methods provide test suite generation techniques with guaranteed fault coverage under certain hypotheses about the system under test (SUT). While they have always been considered as an interesting research field, their practical applicability has often been questioned, due to the unmanageable test suite sizes to be applied when wishing to achieve full fault coverage under realistic hypotheses. This situation has changed during the last years, since it could be shown that equivalence class partition techniques could be applied to industrial-size SUTs.In this project, we plan to exploit these recent results in the context of property-oriented testing (POT). In contrast to general model-based testing (MBT), POT focuses on the thorough test of single SUT requirements only. As a consequence, POT does not require complete models describing the expected behavior of the SUT but can be based on (1) specifications stated in temporal logics, (2) partial models, or (3) abstracted models. Therefore, the costs for enabling SUT verification by POT are usually lower than the costs for a full MBT approach. On the downside, there is currently no comprehensive theory of completeness for testing against properties. It is therefore a main objective of this project to elaborate new approaches to POT allowing to generate test suites with guaranteed fault coverage. This will be achieved by means of a mixed approach involving both models and temporal logics specifications. To cope with the state space complexity of the models, symbolic methods will be applied, where concrete inputs, outputs, and SUT states are replaced by first-order formulas identifying input, output, and state classes. As an additional challenge, we will investigate POT in the context of autonomous systems. Since POT has close links to MBT, a supporting work package will focus on complete model-based testing theories for systems with infinite domains for inputs, internal states, and outputs. A further work package will investigate the problem of configurable systems as in product line testing and extend the existing equivalence class theory to configuration parameters. The CPOT-SM research results will be implemented as an experimental prototype in an existing industrial-strength MBT tool. The elaborated methods will be validated in this tool framework in case studies to be conducted with international research partners. The case studies cover the fields avionics, railways, autonomous vehicles, and robotics.