Non-Asymptotic Statistical Similarity Measures

点击翻译按钮获取中文摘要

The similarity measures investigated in this proposal can be intuitively understood as measures for a distance. In daily life, the distance between two points is usually a physical length. For example, a flat might be located 3km from the city center. However, this is not the only way of measuring distances. The same flat might also be described as being "15min" or "5 bus stops" from the city center.Depending on the use-case, distance measures can also be significantly more abstract. Sticking with the above example, assume that someone is looking for a flat in a city. Apart from the distance to the city center, there are other important criteria such as the size of the flat or the monthly rent. When formulating the "flat-hunting problem'' mathematically, it is convenient to combine all these aspects into a single function that defines a distance from an ideal flat. From all available flats, one then picks the one that minimizes this distance, i.e., comes closest to the ideal.Solving decision making problems by minimizing a suitable distance is a conceptually elegant and powerful method. In practice, however, the problem of how to choose the correct distance measure arises. In our example, the person looking for a flat does not think in terms of abstract distances, but in terms of goals and constraints: they need a flat of a minimum size, at a maximum monthly rent, as close as possible to a supermarket, bus stop, school, etc. Mathematically, this corresponds to a constrained optimization problem. This formulation is often less elegant, but allows for a clear interpretation of the solution. In a nutshell, the aim of this project is to combine both approaches in order to get the best of both worlds. More specifically, a systematic method will be developed that allows to construct distance measures so that the minimization of a distance is equivalent to the solution of a well-defined optimization problem.The proposed research is focused on problems of statistical inference, i.e. obtaining information about the state of a system -- in an optimal manner -- from noisy observations. The corresponding distances are defined on the space of probability distributions. Existing statistical distances, such as the Kullback--Leibler divergence or the alpha divergence, are either based on infinite (asymptotic) sample sizes or axioms, so that they require strong assumptions and do not lend themselves to a clear interpretation. A successful completion of this project will make it possible to construct statistical distances from well-defined, non-asymptotic inference problems. This will in turn allow for new insights into open problems in statistical signal processing and related fields and for extensions of existing results based on transparent theoretical foundations.