Flexible Tails for Normalising Flows

对决策数据进行有原则的分析，无论是人工主导还是自动化，都需要处理不确定性。成熟的概率论领域为处理这类性质的问题提供了丰富的理论支持。当概率被用来描述和推理一个系统时，特定的假设被称为概率模型。概率建模的中心目标是成功地推断出感兴趣的量的不确定性是如何分布的，称为概率分布。这项任务在许多表现出随机性的应用领域中至关重要，例如面部识别等机器学习应用或预测下一次洪水可能发生的极值估计。概率分布的规定既灵活又便于估计，在这些领域具有明显的价值。规范化流（NF）是一种表达概率分布的机制，最近已经产生了很多研究兴趣。与标准的参数假设不同，NF的中心思想是将潜在复杂的目标分布表示为更简单的基础分布的变换。结果分布的结构由基础分布表示的随机性和变换的形式控制。通过要求变换是平滑的和可逆的，除了从NF中有效地抽取样本之外，还可以通过众所周知的变量变化公式精确地评估NF密度。最近，NF模型已成功地应用于具有挑战性的学习任务。这些应用程序包括生成逼真的面部图像和复杂物理过程的建模。尽管NF在直接执行密度估计时取得了成功，但在应用于贝叶斯推断时仍然存在重大问题，这通常发生在未观察到感兴趣的数量时。NF的一个已知问题是它们无法表示具有重尾的分布。这个问题的当前解决方案引入了重尾基分布。在我们的工作中，我们提出通过均匀基分布的变换来捕获重尾分布。我们的建议有明显的技术优势，目前的方法，可以依靠现有的极值文献的理论支持。这一研究方向有可能为NF模型在重要极值问题中的全新应用开辟道路。此外，我们将研究是否足够灵活的尾巴也可以提高贝叶斯推理与NF。这样的改进将为许多重要的应用提供广泛的益处。

Principled analysis of data for decision making, whether human led or automated, requires dealing with uncertainty. The mature field of probability theory provides a wealth of theoretical support for dealing with problems of this nature. When probability is used to describe and reason about a system, the specific assumptions are referred to as probabilistic models. The central aim of probabilistic modelling is to successfully infer how the uncertainty about quantities of interest is distributed, known as probability distributions. This task is critical in many application domains that exhibit randomness, such as machine learning applications like facial recognition or the estimation of extreme values like predicting when the next flood may occur. The specification of probability distributions that are both flexible and convenient to estimate has clear value in these fields. Normalising flows (NFs) are one such mechanism for expressing probability distributions that have generated much recent research interest. Rather than standard parametric assumptions, the central idea of NFs is to represent a potentially complex target distribution as the transformation of a simpler base distribution. The structure of the resulting distribution is controlled by both the randomness expressed by the base distribution and the form of the transformation. By requiring that the transformation is smooth and invertible, one can evaluate the NF density exactly by the well known change of variable formula in addition to efficiently drawing samples from the NF. Recently, NF models have been successfully applied to challenging learning tasks. These include application to generating realistic images of faces and the modelling of complex physical processes. Despite the successes of NFs when performing density estimation directly, significant issues remain when applied to Bayesian inference, which commonly occurs when the quantity of interest is unobserved. One known issue with NFs is their inability to represent distributions with heavy tails. A current solution to this problem introduces a heavy tailed base distribution. In our work, we instead propose capturing heavy tailed distributions via a transformation of uniform base distributions. Our proposal has clear technical advantages over current approaches and can rely on existing extreme value literature for theoretical support. This direction of research has the potential to open up entirely new applications of NF models to important extreme value problems. Additionally, we will investigate whether sufficiently flexible tails could also improve Bayesian inference with NFs. Such an improvement would deliver wide ranging benefit to many important applications.