Statistical Decision Making: Advances in Methods and Techniques

该项目属于EPSRC的人工智能和机器人研究领域的福尔斯。其目的是改进决策算法和贝叶斯机器学习方法，以完成控制自主代理（物理或基于软件）的任务。在决策的背景下，贝叶斯建模提供了一种强大的方式来量化一个人对世界的信念。决策代理人对周围环境有准确的信念，对他们的行动结果有一个良好的校准概念，可以超越那些不考虑不确定性的代理人。例如，这些代理人可以更有效地平衡探索不同行为的影响或利用现有的如何行动的知识的需要，以实现其目标。这些代理人也可能对他们的行动所带来的风险更加敏感，这对这种部署的安全性具有重要影响。智能体经常接收高维的感觉信息，并收集大量的观察结果。在这些环境中，贝叶斯建模技术可能会因缺乏计算易处理性而受到影响，并且经常因神经网络的灵活性和速度而被回避。该项目的最初重点之一是开发贝叶斯深度学习方法用于这种环境，该方法保留了深度神经网络的代表性能力，同时还保留了准确的预测不确定性，并且易于优化。这个项目的第二个重点是卓有成效地使用这些新的贝叶斯方法结合决策算法，以提高自主代理的控制。实践者定义了代理在给定任务上的性能的一些客观度量，之后代理以自我导向的方式探索其环境，目标是学习如何采取行动以最大化其目标。重要的研究问题包括如何提高智能体的样本效率（即智能体如何有效地收集经验以最大化其目标）、对风险的敏感性以及通过考虑环境中的因果结构来提前计划行动序列的能力。近年来，此类智能体在一系列重要任务中已经达到甚至超过了人类水平的表现。利用神经网络的能力来学习环境的有效表示，使我们能够解决一类更具挑战性的控制问题，迄今为止，这些问题很难用传统方法建模。负责和有效的自动化现实世界的问题的范围提供了进一步的动机，这些方法的发展。

This project falls within the EPSRC's Artificial Intelligence and Robotics research area. The aim is to improve decision-making algorithms and Bayesian machine learning methods, for the task of controlling autonomous agents (either physical or software-based). In the context of decision making, Bayesian modelling provides a robust way of quantifying one's beliefs about the world. Decision-making agents with accurate beliefs about their surrounding environment, and a well-calibrated notion of uncertainty about the results of their actions, can out-perform agents which do not account for uncertainty in this way. For instance, such agents can more effectively balance the need to explore the effects of different behaviours or capitalise on existing knowledge of how to act, to pursue their objectives. These agents can also be more sensitive to the risk that their actions carry, with important implications for safety of such deployments. Agents often receive high-dimensional sensory information and collect vast numbers of observations. In these settings, Bayesian modelling techniques may suffer from a lack of computational tractability and are often eschewed for the flexibility and speed of neural networks. One of the initial focuses of this project is to develop Bayesian deep learning methods for use in this setting, which retain the representational ability of deep neural networks, while also retaining accurate predictive uncertainty, and being readily optimisable. The second focus of this project is to fruitfully use these new Bayesian methods in conjunction with decision-making algorithms, to improve the control of autonomous agents. The practitioner defines some objective measure of the agent's performance on a given task, after which the agent is left to explore its environment in a self-directed way, with the goal of learning how to act to maximise its objective. Important research problems include how to improve the agent's sample efficiency (that is, how effectively the agent can gather experience to maximise its objective), sensitivity to risk, and ability to plan sequences of actions in advance, by accounting for causal structures in the environment.In recent years, such agents have already reached or even surpassed human-level performance in a range of non-trivial tasks. Leveraging the ability of neural networks to learn effective representations of the environment gives us purchase on an ever more challenging class of control problems, which have hitherto been difficult to model with traditional methods. The scope for responsible and effective automation of real-world problems provides further motivation for the development of these methods.