An information theoretic approach to autonomous learning of embodied agents

理想情况下，自主学习系统应该能够收集足够的关于自身和环境的信息来解决给定的任务。这显然意味着它们不应该依赖于人类监督者。因此，主要问题是：如果系统只是偶尔接收到是或否反馈，那么它如何能够对自身和环境不具备先验知识的系统识别其任务并学会最佳解决它。这种类型的系统适用于未知和动态环境中的操作。为了能够在这样的环境中发挥作用，系统还必须检测干扰，比如一个轮子堵塞，并找到克服这种损害的方法。由于当今机器人系统的复杂性已经挑战了传统的编程方法，自主学习的能力将变得越来越重要。在这个项目中，我们将推导出有助于在人工系统中实现上述能力的数学方法。我们主要关注的是理论建设和基础研究。我们认为，在自主学习的背景下，我们必须考虑三个主要方面。首先，学习系统必须有一种内在的动机去探索它的本体和环境；探索是这个项目的核心要素。其次，探索必须以身体和环境的物理约束为指导。并不是每一个动作在每一个时间点都是可能的。自主识别哪些动作是有效的，何时有效，这将大大减少搜索空间，从而提高学习效率。第三，内部动机必须与外部反馈信号有效结合。在此背景下，我们认为具身自主学习领域需要信息论和信息几何的新方法来实现自主学习的下一大步。在这个项目中，我们将研究自主学习的理论基础，并在虚拟机器人系统中验证它们。

Ideally, autonomously learning systems should be able to gather enough information about themselves and their environment to solve a given task. This explicitly means that they should not rely on a human supervisor. The main question therefore is: How can a system that ist not equipped with a priori knowledge about itself and the environment identify its task and learn to optimally solve it, if it only occasionally receives a yes-no-feedback. This type of system is desired for operations in unknown and dynamic environments. To be able to function in such environments, the system must also detect disturbances, like a blockage of one wheel, and find a way to overcome such an impairment. The ability of autonomous learning will become more and more essential as the complexity of todays robotic systems already challenge the classical programming approach. In this project, we will derive the mathematical methods which will help to realise the mentioned capacities in artificial systems. Our main concerns are theory construction and basic research. We believe that there are three main aspects that we have to consider in the context of autonomous learning. First, the learning system must have a form of internal motivation to explore its body and environment; exploration is a central element of this project. Second, the exploration must be guided by the physical constraints of the body and environment. Not every action is possible at every point in time. To autonomously recognise which actions are effective and when will improve the learning as it reduces the search space significantly. Third, the internal motivation must be combined effectively with an external feedback signal. In this context, we believe that the field of embodied autonomous learning requires new methods from information theory and information geometry to make the next big step towards autonomy. In this project, we will investigate the theoretical foundations of autonomous learning and validate them in virtual robotic systems.