RoboTest: Systematic Model-Based Testing and Simulation of Mobile Autonomous Robots

移动的和自主机器人在工业和更广泛的社会中发挥着越来越重要的作用;从无人驾驶汽车到家庭辅助，潜在的应用是众多的。英国政府将机器人技术确定为引领我们未来经济增长的关键技术（tinyurl.com/q8bhcy7）。然而，他们已经认识到，自主机器人是复杂的，通常在不断变化的环境中运行（tinyurl.com/o2u2ts7）。我们如何能确信它们能按要求发挥有用的功能，但又是安全的呢？使用测试来检查正确性和安全性是标准做法。机器人的软件开发实践通常包括在机器人构建之前在模拟中进行测试，然后测试实际机器人。模拟有几个好处：我们可以尽早测试，测试执行更便宜，更快。例如，仿真不需要机器人进行物理移动。然而，仍然需要使用真实的机器人进行测试，因为我们不能确定仿真是否能够捕获硬件和环境的所有重要方面。在当前的场景中，测试生成通常是手动的;这使得测试昂贵且不可靠，并引入了延迟。手动生成测试很容易出错，并且可能导致产生错误结论的测试。如果测试错误地指出机器人有故障，那么开发人员必须进行调查，这需要额外的成本和时间。如果测试不正确地指出机器人的行为符合预期，则可能会释放有故障的系统。如果没有系统的方法，测试也可能识别不可行的环境;这样的测试不能用于真实的机器人。更糟糕的是，手动测试生成限制了生成的测试数量。所有这些都影响了机器人软件的成本和质量，并且与其他安全关键领域（如运输行业）的当前实践形成鲜明对比。然而，技术的翻译并不是微不足道的。例如，缺乏驾驶员来纠正错误或应对不可预见的情况，导致自动驾驶汽车的工作条件要大得多。另一个例子是概率算法，它使机器人的行为不确定，因此，在测试中很难重复，更难判断正确与否。 我们将解决所有这些问题与新的自动测试生成技术的移动的和自主机器人。为了使用我们的技术，RoboTest测试人员使用模拟和实现设计中已经采用的熟悉符号构建机器人模型。之后，RoboTest测试人员无需花时间设计模拟场景，只需按下按钮即可生成测试。使用RoboTest，测试成本更低，因为它需要更少的时间，并且更有效，因为RoboTest测试人员可以使用更多的测试，特别是在使用模拟时。为了执行测试，RoboTest测试人员可以从采用各种编程方法的几个模拟器中进行选择。测试的执行也遵循一个按钮的推动。还有一个按钮将模拟转换为部署测试。因此，RoboTest测试人员可以将部署测试的结果追溯到模拟和原始模型。因此，RoboTest测试人员在了解仿真和真实的世界之间的现实差距方面处于有利地位。RoboTest测试人员知道测试的判决是正确的，并了解测试实现了什么;例如，可以保证找到已识别类的错误。因此，RoboTest测试人员可以回答一个非常困难的问题：我们测试得足够了吗？总之，RoboTest将把移动的和自主机器人的测试转移到一个坚实的基础上。RoboTest将使测试在人力方面更加高效和有效，从而实现更长期的成本降低。

Mobile and autonomous robots have an increasingly important role in industry and the wider society; from driverless vehicles to home assistance, potential applications are numerous. The UK government identified robotics as a key technology that will lead us to future economic growth (tinyurl.com/q8bhcy7). They have recognised, however, that autonomous robots are complex and typically operate in ever-changing environments (tinyurl.com/o2u2ts7). How can we be confident that they perform useful functions, as required, but are safe? It is standard practice to use testing to check correctness and safety. The software-development practice for robotics typically includes testing within simulations, before robots are built, and then testing of the actual robots. Simulations have several benefits: we can test early, and test execution is cheaper and faster. For example, simulation does not require a robot to move physically. Testing with the real robots is, however, still needed, since we cannot be sure that a simulation captures all the important aspects of the hardware and environment.In the current scenario, test generation is typically manual; this makes testing expensive and unreliable, and introduces delays. Manual test generation is error-prone and can lead to tests that produce the wrong verdict. If a test incorrectly states that the robot has a failure, then developers have to investigate, with extra cost and time. If a test incorrectly states that the robot behaves as expected, then a faulty system may be released. Without a systematic approach, tests may also identify infeasible environments; such tests cannot be used with the real robot. To make matters worse, manual test generation limits the number of tests produced. All this affects the cost and quality of robot software, and is in contrast with current practice in other safety-critical areas, like the transport industry, which is highly regulated. Translation of technology, however, is not trivial. For example, lack of a driver to correct mistakes or respond to unforeseen circumstances leads to a much larger set of working conditions for an autonomous vehicle. Another example is provided by probabilistic algorithms, which make the robot behaviour nondeterministic, and so, difficult to repeat in testing and more difficult to characterise as correct or not. We will address all these issues with novel automated test-generation techniques for mobile and autonomous robots. To use our techniques, a RoboTest tester constructs a model of the robot using a familiar notation already employed in the design of simulations and implementations. After that, instead of spending time designing simulation scenarios, the RoboTest tester, with the push of a button, generates tests. With RoboTest, testing is cheaper, since it takes less time, and is more effective, because the RoboTest tester can use many more tests, especially when using a simulation.To execute the tests, the RoboTest tester can choose from a few simulators employing a variety of approaches to programming. Execution of the tests also follows the push of a button. Yet another button translates simulation to deployment tests. So, the RoboTest tester can trace back the results from the deployment tests to the simulation and the original model. So, the RoboTest tester is in a strong position to understand the reality gap between the simulation and the real world.The RoboTest tester knows that the verdicts for the tests are correct, and understands what the testing achieves; for example, it can be guaranteed to find faults of an identified class. So, the RoboTest tester can answer the very difficult question: have we tested enough?In conclusion, RoboTest will move the testing of mobile and autonomous robots onto a sound footing. RoboTest will make testing more efficient and effective in terms of person effort, and so, achieve longer term reduced costs.