Computer vision and deep learning: Bayesian Reinforcement Learning

拟议的Engage赠款将探索贝叶斯强化学习(BRL)的效用，并将其集成到Sightline的高性能计算(HPC)框架中。强化学习(RL)是一类机器学习(ML)方法，在该方法中，软件代理与未知环境交互，目的是学习或寻找策略，以优化某些性能度量。基本模型包括马尔可夫决策过程及其变体。BRL通常是竞争性的和无监督的，其目标是试图通过产生X的概率分布来估计随机变量X。X是通过相关随机变量Y的观测或样本来推断的。复合困难是与Y的测量或样本相关的噪声。尽管BRL经过深思熟虑并设计良好，但尚未被广泛应用或采用。许多人认为，尽管BRL已经取得了成功，但它仍然处于初级阶段，这一方面与从应用角度接近机器学习的Sightline直接相关。预计，将强大的政策近似模型(如与深度概率网络相关联的模型)与BRL方法相结合，可以进一步促进与政策优化相关的更好的勘探-开采权衡。这一接洽机会将为Sightline提供另一个ML产品，以增加其分析工具套件，并探索BRL未得到充分利用和开发的潜力。

The proposed Engage grant will explore the utility of Bayesian Reinforcement Learning (BRL) and integrate it in to the High Performance computing (HPC) framework of Sightline. In general Reinforcement Learning (RL), is a class of machine learning (ML) methods in which a software agent interacts with an unknown environment, with the goal of learning or finding a policy, to optimize some performance metric. Underlying models include Markov Decision Processes and their variants. BRL is typically competitive and unsupervised with the objective of attempting to estimate a random variable X by producing a probability distribution for X. X is inferred through observations or samples of a related random variable Y. Compounding difficulty is noise associated with measurements or samples of Y. Although well-thought-out and well-designed, BRL has not been widely applied or adopted. Many believe that BRL is still in its infancy in spite of demonstrated successes and this aspect is of direct interest to Sightline who are approaching machine learning from the application perspective. It is anticipated that the combination of powerful models for policy approximation such as those associated with deep probabilistic networks in combination with a BRL approach can further facilitate better exploration-exploitation trade-offs associated with policy optimization. This Engage opportunity will provide Sightline with another ML offering to add to its suite of analytics tools and explore the underutilized and untapped potential of BRL.