Flexible Tails for Normalising Flows

点击翻译按钮获取中文摘要

Principled analysis of data for decision making, whether human led or automated, requires dealing with uncertainty. The mature field of probability theory provides a wealth of theoretical support for dealing with problems of this nature. When probability is used to describe and reason about a system, the specific assumptions are referred to as probabilistic models. The central aim of probabilistic modelling is to successfully infer how the uncertainty about quantities of interest is distributed, known as probability distributions. This task is critical in many application domains that exhibit randomness, such as machine learning applications like facial recognition or the estimation of extreme values like predicting when the next flood may occur. The specification of probability distributions that are both flexible and convenient to estimate has clear value in these fields. Normalising flows (NFs) are one such mechanism for expressing probability distributions that have generated much recent research interest. Rather than standard parametric assumptions, the central idea of NFs is to represent a potentially complex target distribution as the transformation of a simpler base distribution. The structure of the resulting distribution is controlled by both the randomness expressed by the base distribution and the form of the transformation. By requiring that the transformation is smooth and invertible, one can evaluate the NF density exactly by the well known change of variable formula in addition to efficiently drawing samples from the NF. Recently, NF models have been successfully applied to challenging learning tasks. These include application to generating realistic images of faces and the modelling of complex physical processes. Despite the successes of NFs when performing density estimation directly, significant issues remain when applied to Bayesian inference, which commonly occurs when the quantity of interest is unobserved. One known issue with NFs is their inability to represent distributions with heavy tails. A current solution to this problem introduces a heavy tailed base distribution. In our work, we instead propose capturing heavy tailed distributions via a transformation of uniform base distributions. Our proposal has clear technical advantages over current approaches and can rely on existing extreme value literature for theoretical support. This direction of research has the potential to open up entirely new applications of NF models to important extreme value problems. Additionally, we will investigate whether sufficiently flexible tails could also improve Bayesian inference with NFs. Such an improvement would deliver wide ranging benefit to many important applications.