Statistical machine learning for efficient hidden parameter estimation and decision making

现实世界的问题通常被认为是黑盒系统，因为我们可以观察输出，但几乎没有关于输入和内部计算的信息。因为我们不能直接“打开盒子”，所以我们求助于这些值的估计。然而，这些系统的概率性质限制了确定性估计算法的能力。因此，我们采用贝叶斯方法来估计输出的概率密度。我们可以进一步尝试通过估计产生密度的潜在变量来对密度进行参数化。这些可以被视为我们的黑匣子的输入。这些问题中的大多数样本数据都是令人难以置信的高维数据。不幸的是，高维数据的密度估计导致了精度和效率之间的权衡。这个问题也被称为“维度诅咒”。利用深度学习的力量来估计密度是一个新的研究领域。神经网络的巨大解释能力与通过现代硬件的有效实施相结合，显示出有希望准确和高效地估计这些密度。我们的研究旨在通过探索最新的贝叶斯机器学习技术来提高密度和潜在变量估计的精度和效率。更具体地说，通过第一年，我们将探索将变分自动编码器与神经密度估计器集成起来，用于变分推理和潜变量估计。通过第二年和第三年，我们目前计划比较类似于神经密度估计的方法，如自回归流动。此外，我们将探索从地球科学到生物学的广泛应用，以充分测试我们的技术。我们预计该项目将提高参数估计的效率和可靠性，并提高复杂工业应用的决策过程，特别是高维应用。与大多数具有上述黑盒系统的问题相比，具有已知输入和内部计算的真实世界问题只是一个很小的子集。因此，这使得我们的研究的应用程序是深远的。我们的研究与EPSRC的相关性是通过开发新的计算方法来提供跨处理复杂数据的所有领域的解决方案。

Real-world problems are often considered black-box systems, as we can observe the output but have little information about the inputs and inner computations. Since we can not directly "open the box", we resort to estimations of these values. However, the probabilistic nature of these systems limits the ability of deterministic estimation algorithms. Thus, we take a Bayesian approach and estimate the probability density of the output. We can further attempt to parameterise this density by estimating the latent variables that produce the density. These can be considered the input of our black box. The sample data in most of these problems are incredibly high-dimensional. Unfortunately, density estimation of high-dimensional data results in a tradeoff between accuracy and efficiency. This problem is also known as the "curse of dimensionality". Leveraging the power of deep learning to estimate density is a novel area of research. The vast explanatory power of neural networks combined with efficient implementation through modern hardware shows promise to accurately and efficiently estimate these densities. Our research aims to improve the accuracy and efficiency of density and latent variable estimation through exploring the state of the art Bayesian machine learning techniques. More specifically through year one, we will explore integrating Variational autoencoders with neural density estimators for variational inference and latent variable estimation. Through years two and three we currently plan to compare methods similar to neural density estimation, such as auto-regressive flows. Furthermore, we will explore a wide range of applications, from geoscience to biology, to fully test our techniques. We expect the project to improve the efficiency and reliability of parameter estimation and the decision-making process for complex industrial applications, particularly high dimensional. Real-world problems with known input and internal computations are a small subset compared to most that have the aforementioned black box systems. As such, this allows the applications of our research to be far-reaching.The relevance of our research to EPSRC is through the development of novel computational methods that will provide solutions across all fields handling complex data.