Can Direct Latent Model Learning Solve Linear Quadratic Gaussian Control?
Can Direct Latent Model Learning Solve Linear Quadratic Gaussian Control?
复制标题
DOI：
10.48550/arxiv.2212.14511
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Yi Tian;K. Zhang;Russ Tedrake;S. Sra
中科院分区：
文献类型：
--
作者：
Yi Tian;K. Zhang;Russ Tedrake;S. Sra
我们研究从潜在的高维观察中学习状态表示的任务，其目标是控制未知的部分可观察系统。我们追求一种直接的潜在模型学习方法，其中通过预测与规划直接相关的数量（例如成本）来学习某些潜在状态空间中的动态模型，而无需重建观察结果。特别是，我们专注于一种直观的成本驱动状态表示学习方法，用于解决线性二次高斯（LQG）控制，这是最基本的部分可观察控制问题之一。作为我们的主要结果，我们建立了使用直接学习的潜在模型找到接近最优状态表示函数和接近最优控制器的有限样本保证。据我们所知，尽管取得了各种经验上的成功，但在这项工作之前，尚不清楚这种成本驱动的潜在模型学习器是否享有有限样本保证。我们的工作强调了预测多步骤成本的价值，这是我们理论的关键思想，尤其是对于学习状态表示具有经验价值的思想。
We study the task of learning state representations from potentially high-dimensional observations, with the goal of controlling an unknown partially observable system. We pursue a direct latent model learning approach, where a dynamic model in some latent state space is learned by predicting quantities directly related to planning (e.g., costs) without reconstructing the observations. In particular, we focus on an intuitive cost-driven state representation learning method for solving Linear Quadratic Gaussian (LQG) control, one of the most fundamental partially observable control problems. As our main results, we establish finite-sample guarantees of finding a near-optimal state representation function and a near-optimal controller using the directly learned latent model. To the best of our knowledge, despite various empirical successes, prior to this work it was unclear if such a cost-driven latent model learner enjoys finite-sample guarantees. Our work underscores the value of predicting multi-step costs, an idea that is key to our theory, and notably also an idea that is known to be empirically valuable for learning state representations.