Statistical machine learning for efficient hidden parameter estimation and decision making

现实世界的问题通常被认为是黑盒系统，因为我们可以观察到输出，但几乎没有关于输入和内部计算的信息。由于我们不能直接“打开盒子”，我们求助于这些值的估计。然而，这些系统的概率性质限制了确定性估计算法的能力。因此，我们采用贝叶斯方法来估计输出的概率密度。我们可以进一步尝试通过估计产生密度的潜在变量来参数化这个密度。这些可以看作是我们黑盒的输入。这些问题中的大多数样本数据都是高维的。不幸的是，高维数据的密度估计会导致准确性和效率之间的权衡。这个问题也被称为“维度的诅咒”。利用深度学习的力量来估计密度是一个新的研究领域。神经网络的强大解释能力与现代硬件的高效实现相结合，显示出准确有效地估计这些密度的希望。我们的研究旨在通过探索最先进的贝叶斯机器学习技术来提高密度和潜在变量估计的准确性和效率。更具体地说，在第一年，我们将探索将变分自编码器与神经密度估计器相结合，用于变分推理和潜在变量估计。在第二和第三年，我们目前计划比较类似于神经密度估计的方法，例如自回归流。此外，我们将探索从地球科学到生物学的广泛应用，以充分测试我们的技术。我们期望该项目能够提高复杂工业应用，特别是高维应用的参数估计和决策过程的效率和可靠性。与具有上述黑盒系统的大多数问题相比，具有已知输入和内部计算的实际问题只是一个很小的子集。因此，这使得我们的研究的应用是深远的。我们的研究与EPSRC的关联是通过开发新颖的计算方法，这些方法将为处理复杂数据的所有领域提供解决方案。

Real-world problems are often considered black-box systems, as we can observe the output but have little information about the inputs and inner computations. Since we can not directly "open the box", we resort to estimations of these values. However, the probabilistic nature of these systems limits the ability of deterministic estimation algorithms. Thus, we take a Bayesian approach and estimate the probability density of the output. We can further attempt to parameterise this density by estimating the latent variables that produce the density. These can be considered the input of our black box. The sample data in most of these problems are incredibly high-dimensional. Unfortunately, density estimation of high-dimensional data results in a tradeoff between accuracy and efficiency. This problem is also known as the "curse of dimensionality". Leveraging the power of deep learning to estimate density is a novel area of research. The vast explanatory power of neural networks combined with efficient implementation through modern hardware shows promise to accurately and efficiently estimate these densities. Our research aims to improve the accuracy and efficiency of density and latent variable estimation through exploring the state of the art Bayesian machine learning techniques. More specifically through year one, we will explore integrating Variational autoencoders with neural density estimators for variational inference and latent variable estimation. Through years two and three we currently plan to compare methods similar to neural density estimation, such as auto-regressive flows. Furthermore, we will explore a wide range of applications, from geoscience to biology, to fully test our techniques. We expect the project to improve the efficiency and reliability of parameter estimation and the decision-making process for complex industrial applications, particularly high dimensional. Real-world problems with known input and internal computations are a small subset compared to most that have the aforementioned black box systems. As such, this allows the applications of our research to be far-reaching.The relevance of our research to EPSRC is through the development of novel computational methods that will provide solutions across all fields handling complex data.