Extracting Latent State Representations with Linear Dynamics from Rich Observations-猫眼课题宝

DOI：

发表时间：

2020-06

期刊：

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Abraham Frandsen;Rong Ge

最近，许多强化学习技术被证明在线性动力学的简单情况下具有可证明的保证，特别是在线性二次调节器等问题中。然而，在实践中，许多强化学习问题试图直接从丰富的高维表示（如图像）中学习策略。即使在正确的潜在表示（如位置和速度）中存在线性的潜在动态，丰富的表示也可能是非线性的，并且可能包含不相关的特征。在这项工作中，我们研究了一个模型，其中存在一个隐藏的线性子空间，其中动态是线性的。对于这样的模型，我们给出了一个有效的算法提取线性动力学的线性子空间。然后，我们将我们的想法扩展到提取非线性映射，并在具有丰富观测的简单设置中实证验证了我们的方法的有效性。

Recently, many reinforcement learning techniques were shown to have provable guarantees in the simple case of linear dynamics, especially in problems like linear quadratic regulators. However, in practice, many reinforcement learning problems try to learn a policy directly from rich, high dimensional representations such as images. Even if there is an underlying dynamics that is linear in the correct latent representations (such as position and velocity), the rich representation is likely to be nonlinear and can contain irrelevant features. In this work we study a model where there is a hidden linear subspace in which the dynamics is linear. For such a model we give an efficient algorithm for extracting the linear subspace with linear dynamics. We then extend our idea to extracting a nonlinear mapping, and empirically verify the effectiveness of our approach in simple settings with rich observations.

Extracting Latent State Representations with Linear Dynamics from Rich Observations

文献摘要

相似文献