RI: Small: Approximate Inference for Planning and Reinforcement Learning

随机规划，即在环境包含不确定性时控制系统的问题，在许多行业和科学领域中无处不在。一些重要的例子包括控制风力发电场和操纵复杂的机器人系统。在许多情况下，这些问题必须得到解决，即使没有世界如何行为的精确模型。例如，当施加一些物理力时，我们可能不知道机器人在移动过程中是否会过弯或打滑。这就要求智能体将联合收割机对世界的学习与规划结合起来。这些学习和规划问题在计算上具有挑战性，开发解决这些问题的工具仍然是人工智能研究的主要目标。概率推理提供了一个通用的数学框架，可以捕捉潜在的观察结果和感兴趣的事件之间的联系。一项令人兴奋的工作旨在通过概率推理的透镜，通过将假设行为和这些行为的假设结果与潜在的未来效用相关联来开发规划算法。该项目将从这个角度开发新的机器学习和规划算法。重点是在推理过程中近似的需要和规划质量的近似的影响之间的相互作用。该项目将在多个领域的一系列问题中评估人工智能解决方案。该项目探讨了人工智能规划，机器人最优控制和强化学习中的问题，这些问题共享一些技术核心，但通常被单独研究。更具体地说，该项目将开发用于随机规划、机器学习中的不确定性量化和基于模型的强化学习的算法。在解决这些优化问题时，包括离散量和连续量的混合环境提供了额外的计算挑战。为了设计有效的规划算法，该项目将开发和利用关于可微符号概率推理及其与消息传递推理算法的联系的想法。此外，将通过将多个时间尺度的过渡模型集成到推理算法中来实现长期搜索。为了设计有效的强化学习算法，该项目将首先开发新的方法，用于使用深度神经网络进行不确定性量化，这将有助于模型学习。反过来，将模型不确定性与规划算法相结合将产生新的基于模型的强化学习方法。这些算法将被细化为真实的时间控制，使机器人系统中的应用程序。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Stochastic planning, the problem of controlling a system when the environment includes uncertainty, is ubiquitous across many industries and scientific areas. Some important examples include controlling wind farms for energy generation and manipulating a complex robotic system. In many cases, these problems have to be solved even though there is no precise model of how the world behaves. For example, when applying some physical force, we may not know whether a robot will overshoot a corner or slip during its movement. This requires the agent to combine learning about the world with planning. These learning and planning problems are computationally challenging and developing tools for solving them is still a major goal for AI research. Probabilistic inference provides a general mathematical framework that captures connections between potential observations and events of interest. An exciting line of work aims to develop planning algorithms through the lens of probabilistic inference, by associating hypothetical actions and hypothetical outcomes of such actions with potential future utility. The project will develop new machine learning and planning algorithms from this perspective. The focus is on the interaction between the need for approximations in the inference process and the implications of approximations for planning quality. Algorithmic solutions will be evaluated in a range of problems across several fields.The project explores problems across AI planning, optimal control in robotics, and reinforcement learning that share some technical core, but have often been studied separately. More specifically the project will develop algorithms for stochastic planning, for uncertainty quantification in machine learning and for model based reinforcement learning. Hybrid environments that include both discrete quantities and continuous quantities provide an additional computational challenge when solving these optimization problems. To devise effective planning algorithms, the project will develop and exploit ideas about differentiable symbolic probabilistic inference and its connection to message passing inference algorithms. In addition, long horizon search will be enabled by integrating transition models for multiple time scales into inference algorithms. To devise effective reinforcement learning algorithms, the project will first develop new methods for uncertainty quantification with deep neural networks that will be instrumental for model learning. In turn, combining model uncertainty with planning algorithms will yield novel model based reinforcement learning methods. These algorithms will be refined for real time control to enable applications in robotic systems. The algorithms will be evaluated on problems from all three fields to guarantee generality and cross fertilization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.