Immersive Visualization of Reinforcement Learning

点击翻译按钮获取中文摘要

Borealis AI is the Royal Bank of Canada (RBC) Institute for Research, pursuing fundamental and applied research in reinforcement learning (RL), deep learning, adversarial machine learning, and other related areas. In addition to applying state of the art methods, they develop new RL techniques specifically designed to address these real-world challenges. One of the fundamental improvements to the framework is by incorporating human knowledge - leveraging a user's abilities can help the system learn faster and better across a wide range of tasks. The long-term goal of this research is to learn from both explicit and implicit user feedback to the system. In order to make progress towards this goal, there are two challenges that must be overcome. First, the non-technical trainer must be able to understand an agent's policy, or way of acting. We hypothesize that the better the trainer understands what the agent does and does not know, the better she will be able to provide assistance. Second, the Borealis AI RL experts must better understand how a trainer's inputs are affecting the algorithm. We hypothesize that moving from the standard black box paradigm to one in which learning is more transparent will allow us to develop new and improved RL methods to better leverage human feedback, as well as to help set the numerous hyper-parameters involved in learning. Borealis AI is looking for help to apply visualization techniques to attack these two critical problems. **The project's goal is to investigate whether immersive visualizations can improve understanding of the RL process for both non-technical and expert users. We will explore visualization techniques in Virtual and Augmented Reality (VR and AR) using the HTC Vive and the Microsoft HoloLens to answer the following questions: 1) What are different ways to visualize the policies that agents use for learning in an immersive environment, which helps a lay audience understand the policies? 2) How can we visualize the algorithm's internal data (e.g., a neural network) in an immersive environment to demonstrate how the knowledge is evolving during the learning process?**