Immersive Visualization of Reinforcement Learning

强化学习的沉浸式可视化

• 批准号: 531581-2018
• 负责人: Maurer, Frank
• 金额: $ 1.82万
• 依托单位: University of Calgary
• 依托单位国家: 加拿大
• 项目类别: Engage Grants Program
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=659076
• 关键词: Immersive; Visualization; Reinforcement; Learning

Borealis AI is the Royal Bank of Canada (RBC) Institute for Research, pursuing fundamental and applied research in reinforcement learning (RL), deep learning, adversarial machine learning, and other related areas. In addition to applying state of the art methods, they develop new RL techniques specifically designed to address these real-world challenges. One of the fundamental improvements to the framework is by incorporating human knowledge - leveraging a user's abilities can help the system learn faster and better across a wide range of tasks. The long-term goal of this research is to learn from both explicit and implicit user feedback to the system. In order to make progress towards this goal, there are two challenges that must be overcome. First, the non-technical trainer must be able to understand an agent's policy, or way of acting. We hypothesize that the better the trainer understands what the agent does and does not know, the better she will be able to provide assistance. Second, the Borealis AI RL experts must better understand how a trainer's inputs are affecting the algorithm. We hypothesize that moving from the standard black box paradigm to one in which learning is more transparent will allow us to develop new and improved RL methods to better leverage human feedback, as well as to help set the numerous hyper-parameters involved in learning. Borealis AI is looking for help to apply visualization techniques to attack these two critical problems. **The project's goal is to investigate whether immersive visualizations can improve understanding of the RL process for both non-technical and expert users. We will explore visualization techniques in Virtual and Augmented Reality (VR and AR) using the HTC Vive and the Microsoft HoloLens to answer the following questions: 1) What are different ways to visualize the policies that agents use for learning in an immersive environment, which helps a lay audience understand the policies? 2) How can we visualize the algorithm's internal data (e.g., a neural network) in an immersive environment to demonstrate how the knowledge is evolving during the learning process?**

Borealis AI是加拿大皇家银行（RBC）研究所，致力于强化学习（RL），深度学习，对抗机器学习和其他相关领域的基础和应用研究。除了应用最先进的方法外，他们还开发了专门用于解决这些现实挑战的新RL技术。对框架的一个根本改进是通过整合人类知识-利用用户的能力可以帮助系统在广泛的任务中更快更好地学习。本研究的长期目标是从系统的显式和隐式用户反馈中学习。为了在实现这一目标方面取得进展，必须克服两个挑战。首先，非技术培训师必须能够理解代理商的政策或行为方式。我们假设，培训师越了解代理人做什么和不知道什么，她就越能够提供帮助。其次，Borealis AI RL专家必须更好地理解训练者的输入如何影响算法。我们假设，从标准的黑盒范式转移到学习更透明的范式，将使我们能够开发新的和改进的RL方法，以更好地利用人类反馈，并帮助设置学习中涉及的众多超参数。Borealis AI正在寻求帮助，应用可视化技术来解决这两个关键问题。** 该项目的目标是调查沉浸式可视化是否可以提高非技术和专家用户对RL过程的理解。我们将使用HTC Vive和Microsoft HoloLens探索虚拟和增强现实（VR和AR）中的可视化技术，以回答以下问题：1）在沉浸式环境中可视化代理用于学习的策略有哪些不同的方法，这有助于外行观众理解策略？2)我们如何可视化算法的内部数据（例如，一个神经网络）在一个沉浸式的环境中，以展示知识是如何在学习过程中演变的？**