Improved learning of reactive, cooperative behaviour through learning-based testing

我研究的长期愿景是使用机器学习来改进分布式系统软件的开发。我的主要想法是修改和扩展合作行为学习的概念，以包括基于学习的测试组件的结果，发现学习的合作行为的负面影响，包括预期的功能，效率和安全性的问题。* 为了学习合作行为，我们将分布式系统的组件视为可以执行操作的代理。学习尝试为这些代理创建控件，以便它们一起实现整个代理组的任务功能。使用基于学习的测试来发现负面后果是一个进化的学习过程，其中测试组件创建与被测系统的交互序列，并基于它们与测试目标的接近程度来评估序列。评估用于学习越来越接近产生测试系统正在寻找的负面后果的定序器。主要的研究挑战是如何联合收割机学习的重点合作行为，试图处理所有可能的情况下，系统可以在与重点学习为基础的测试，找到一个特定的序列的情况下，揭示了一个特定的问题的测试系统。这一挑战的可能解决方案可以针对基于学习的测试的执行方式，概括其结果，但也修改了一般的学习者和异常规则的架构。在早期的工作中，我们已经开发了提前喊架构的代理和这种架构的混合学习方法，结合强化学习与进化学习。一个为游戏《韦诺之战》开发敌方控制AI的概念验证系统表明，最好的学习AI能够击败游戏中相当好的人类创造的中央AI。我们探索了使用行为学习来测试系统的弱点，包括游戏在内的几个应用程序。在所有这些应用程序中，我们的测试系统都能够找到测试系统中的几个弱点，包括效率和安全弱点。在这些工作的基础上，我们将探索我们的想法，结合一般的行为学习与基于学习的测试在两个案例研究中，Wesnoth战役系统的扩展和自动化牛配送中心作为物联网应用的一个例子。这项研究的预期结果是能够创建定制的开放分布式系统，成本低于目前的做法，并具有更高的质量，由于自动化。

The long-term vision for my research is to use machine learning to improve the development of distributed systems software. My key idea is to modify and extend a concept for learning of cooperative behaviour to include the results of a learning-based testing component that finds negative consequences of the learned cooperative behaviour, including problems with intended functionality, efficiency and security. ***To learn cooperative behaviour, we see the components of a distributed system as agents that can perform actions. The learning tries to create controls for these agents, so that they together achieve the functionality the whole group of agents is tasked with.***Using learning-based testing for finding negative consequences is an evolutionary learning process in which the testing component creates interaction sequences with the tested system and evaluates the sequences based on how near they come to a testing goal. The evaluation is used to learn sequencers that come nearer and nearer to creating the negative consequences the testing system is looking for.***The main research challenge is how to combine the focus of learning of cooperative behaviour on trying to deal with all possible situations the system can be in with the focus of learning-based testing on finding one particular sequence of situations revealing a particular problem of the tested system. Possible solutions of this challenge can target the way learning-based testing is performed, generalizing its results, but also modifying the general learner and the architecture with exception rules. In earlier work, we have developed the shout-ahead architecture for agents and a hybrid learning method for this architecture, combining reinforcement learning with evolutionary learning. A proof-of-concept system for developing control AIs for enemy units for the game Battle for Wesnoth showed that the best learned AIs are able to beat the rather good human-created central AI that comes with the game. We explored the usage of learning of behavior to test systems for weaknesses for several applications, including games. In all these applications, our test systems were able to find several weaknesses in the tested systems, including efficiency and security weaknesses.***Building on these works, we will explore our ideas for combining general learning of behaviour with learning-based testing in two case studies, an extension of the system for Battle for Wesnoth and an automated cattle distribution center as an example of Internet of Things applications. The anticipated result of this research is the ability to create customized open distributed systems at lower cost than current practise and with higher quality due to automization.