Variational methods in Bayesian inference

贝叶斯推理是现代统计学的核心技术之一，其形式多种多样。它是提取复杂随机系统及其参数信息的一种重要方法，已成功地应用于许多不同的应用领域，如医学试验分析，生物信息学，法医分析或考古学。它还与图像处理、语音识别和神经网络等相关领域有着密切的联系。因此，贝叶斯推理方法的研究具有重要的现实意义，贝叶斯推理技术的实际实现依赖于计算高维积分的能力。为此目的，已经开发了强大的方法，特别是蒙特-卡罗积分方案，这是非常适合于数值计算的随机算法。虽然这些方法是灵活的和广泛适用的，但在许多情况下，它们受到很大的限制，例如，关于它们的准确性和计算效率。近年来，所谓的变分方法作为一种有前途的替代方法受到越来越多的关注。它是一种确定性近似方案，在某些情况下，允许分析地执行至少部分计算。因此，它是蒙特-卡罗方法的补充，在后者表现不佳的领域可能具有优势。尽管如此，这种方法仍然需要探讨的许多问题，例如关于其精度和可靠性，或开发和优化的高效和灵活的algorithm.Within这方面，该项目在其初始阶段打算获得现有的变分技术的概述，其优点，缺点和适用范围，并将它们与替代方法进行比较。在主要阶段，该项目旨在开发新的变分方法，例如，通过应用相关领域使用的技术（例如，机器学习），结合变分和蒙特卡罗方法，或改进和扩展现有的方法。预计这项工作的结果不仅与该领域正在进行的理论研究有关，而且与贝叶斯方法的实际应用和使用有关，例如，作为数据分析的工具。

Bayesian inference, in its various forms, is one of the core techniques of modern statistics. It is an important method to extract information about complex stochastic systems and their parameters and has successfully been used in many different application areas such as medical trial analysis, bioinformatics, forensic analysis or archealogy. It has also close connections to related fields such as image processing, speech recognition, and neural networks. Research into methods of Bayesian inference is therefore of significant topical interest.The practical implementation of Bayesian inference techniques rests on the ability to calculate high-dimensional integrals. Powerful methods have been developed for this purpose, in particular Monte-Carlo integration schemes, which are stochastic algorithms well suited for numerical computations. Although these methods are flexible and widely applicable, there are many situations where they suffer from significant limitations, e.g., regarding their accuracy and computational efficiency. In recent years, the so-called variational approach has received growing interest as a promising alternative. It is a deterministic approximation scheme that, under certain circumstances, allows to carry out at least some parts of the calculations analytically. It is thus complementary to Monte-Carlo methods and could present advantages in areas where the latter perform badly. Nevertheless, many questions about this approach still need to be explored, for example regarding its precision and reliability, or the development and optimisation of efficient and flexible algorithms.Within this context, this project in its initial stage intends to obtain an overview of the existing variational techniques, their strengths, weaknesses, and areas of applicability, and to compare them to the alternative approaches. In the main stage, the project aims at developing new variational methodology, for example, by applying techniques used in related areas (e.g., machine learning), combining variational and Monte Carlo methods, or improving and extending existing approaches. The results of this work are expected to be relevant not only to the ongoing theoretical research in this field, but also to the practical application and use of Bayesian methods, e.g., as a tool for data analysis.