Active transfer learning with neural networks through human-robot interactions (TRAIN)

在我们的愿景中，自主机器人在工业现场、医疗保健或我们的家中与人类互动。从技术角度来看，所有这些应用领域都要求机器人在执行数千种不同的运动和操作技能时处理大量噪声传感器观测数据。从许多用户的角度来看，手动编程这些技能或使用最新的学习方法（这些方法大多只有专家才能操作）将无法在日常生活任务中使用智能自主系统。在这个项目中，我们的目标是通过考虑人类反馈和指导的深度网络来提高机器人的技能学习。人类教师在人工神经网络中评估不同的迁移学习策略，通过优化现有编码知识来提高新技能的学习。神经网络非常适合这项任务，因为我们可以逐渐增加转移参数的数量，甚至可以在任务特定知识的转移到更深层次编码的抽象特征之间进行转换。为了系统地考虑这一点，我们评估了来自用户实验的主观反馈和生理数据，并制定了评估标准，使以人为本的迁移学习方法得以发展。在两个主要实验中，我们首先研究了用户如何体验迁移学习，然后研究了人类和机器人共享自主性的影响。这将产生一个系统的机器人技能学习框架，以适应用户的需求，例如，通过调整机器人的自主程度来满足外行人的要求。尽管我们评估的学习框架侧重于拟人化机器人手臂的拾取和放置任务，但我们的结果将可转移到广泛的人机交互场景，包括生产和装配中的协作操作任务，以及为康复和家用机器人设计高级控制。

In our vision, autonomous robots are interacting with humans at industrial sites, in health care, or at our homes managing the household. From a technical perspective, all these application domains require that robots process large amounts of data of noisy sensor observations during the execution of thousands of different motor and manipulation skills. From the perspective of many users, programming these skills manually or using recent learning approaches, which are mostly operable only by experts, will not be feasible to use intelligent autonomous systems in tasks of everyday life. In this project, we aim at improving robot skill learning with deep networks considering human feedback and guidance. The human teacher is rating different transfer learning strategies in the artificial neural network to improve the learning of novel skills by optimally exploiting existing encoded knowledge. Neural networks are ideally suited for this task as we can gradually increase the number of transferred parameters and can even transition between the transfer of task specific knowledge to abstract features encoded in deeper layers. To consider this systematically, we evaluate subjective feedback and physiological data from user experiments and elaborate assessment criteria that enable the development of human-oriented transfer learning methods. In two main experiments, we first investigate how users experience transfer learning and then examine the influence of shared autonomy of humans and robots. This will result in a methodical robot skill learning framework that adapts to the users’ needs, e.g., by adjusting the degree of autonomy of the robot to laymen requirements. Even though we evaluate the learning framework focusing on pick and place tasks with anthropomorphic robot arms, our results will be transferable to a broad range of human-robot interaction scenarios including collaborative manipulation tasks in production and assembly, but also for designing advanced controls for rehabilitation and household robots.