Statistical Decision Making: Advances in Methods and Techniques

该项目属于EPSRC的人工智能和机器人研究领域。目的是改进决策算法和贝叶斯机器学习方法，用于控制自主代理（物理或基于软件的）的任务。在决策过程中，贝叶斯模型提供了一种稳健的方法来量化人们对世界的看法。对周围环境有准确信念的决策主体，以及对其行动结果的不确定性有良好校准的概念，可以胜过那些不以这种方式考虑不确定性的主体。例如，这样的代理可以更有效地平衡探索不同行为的影响的需要，或者利用如何采取行动的现有知识来追求他们的目标。这些代理人也可能对其行动所带来的风险更加敏感，这对此类部署的安全性具有重要意义。智能体通常接收高维感官信息并收集大量观察结果。在这些情况下，贝叶斯建模技术可能会受到缺乏计算可追溯性的影响，并且经常被神经网络的灵活性和速度所回避。该项目最初的重点之一是开发用于此设置的贝叶斯深度学习方法，该方法保留了深度神经网络的表示能力，同时保留了准确的预测不确定性，并且易于优化。该项目的第二个重点是将这些新的贝叶斯方法与决策算法结合使用，以改善自主代理的控制。执行者定义了智能体在给定任务上的表现的一些客观度量，之后智能体被留给以自我导向的方式探索其环境，其目标是学习如何采取行动以最大化其目标。重要的研究问题包括如何通过考虑环境中的因果结构来提高智能体的样本效率（即智能体如何有效地收集经验以最大化其目标）、对风险的敏感性以及提前计划行动序列的能力。近年来，这些智能体在一系列重要任务中的表现已经达到甚至超过了人类的水平。利用神经网络学习环境的有效表征的能力，使我们能够解决一类更具挑战性的控制问题，这些问题迄今为止很难用传统方法建模。对现实世界的问题进行负责任和有效的自动化的范围为这些方法的发展提供了进一步的动力。

This project falls within the EPSRC's Artificial Intelligence and Robotics research area. The aim is to improve decision-making algorithms and Bayesian machine learning methods, for the task of controlling autonomous agents (either physical or software-based). In the context of decision making, Bayesian modelling provides a robust way of quantifying one's beliefs about the world. Decision-making agents with accurate beliefs about their surrounding environment, and a well-calibrated notion of uncertainty about the results of their actions, can out-perform agents which do not account for uncertainty in this way. For instance, such agents can more effectively balance the need to explore the effects of different behaviours or capitalise on existing knowledge of how to act, to pursue their objectives. These agents can also be more sensitive to the risk that their actions carry, with important implications for safety of such deployments. Agents often receive high-dimensional sensory information and collect vast numbers of observations. In these settings, Bayesian modelling techniques may suffer from a lack of computational tractability and are often eschewed for the flexibility and speed of neural networks. One of the initial focuses of this project is to develop Bayesian deep learning methods for use in this setting, which retain the representational ability of deep neural networks, while also retaining accurate predictive uncertainty, and being readily optimisable. The second focus of this project is to fruitfully use these new Bayesian methods in conjunction with decision-making algorithms, to improve the control of autonomous agents. The practitioner defines some objective measure of the agent's performance on a given task, after which the agent is left to explore its environment in a self-directed way, with the goal of learning how to act to maximise its objective. Important research problems include how to improve the agent's sample efficiency (that is, how effectively the agent can gather experience to maximise its objective), sensitivity to risk, and ability to plan sequences of actions in advance, by accounting for causal structures in the environment.In recent years, such agents have already reached or even surpassed human-level performance in a range of non-trivial tasks. Leveraging the ability of neural networks to learn effective representations of the environment gives us purchase on an ever more challenging class of control problems, which have hitherto been difficult to model with traditional methods. The scope for responsible and effective automation of real-world problems provides further motivation for the development of these methods.