权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Dynamical systems approach to robust reconstruction of probability distributions of observed data

观测数据概率分布稳健重建的动力系统方法

基本信息

批准号：
415860776
负责人：
Privatdozent Dr. Pavel Gurevich, Ph.D.
金额：
--
依托单位：
Institut für Mathematik
依托单位国家：
德国
项目类别：
Research Grants
财政年份：
2018
资助国家：
德国
起止时间：
2017-12-31 至 2019-12-31
项目状态：
已结题

来源：
https://gepris.dfg.de/gepris/projekt/415860776?language=en
关键词：
Dynamical systems approach robust reconstruction

项目摘要

Our proof-of-concept project aims at constructing a bridge between machine learning and the theory of dynamical systems. More specifically, we develop a dynamical systems paradigm for learning probability distributions of observed data.Artificial neural networks have found a substantial number of applications in recent years due to their ability to represent probability distributions in a parametric way, and effectively learn their parameters from observed data. In our project, we study how the parameters of probability distributions (that are outputs of neural networks) evolve during learning. By considering the limit as the number of data points goes to infinity, we can describe the process of learning the parameters of distributions by a tractable system of differential equations, which we will analyze in detail. As we explain below, these dynamical systems typically possess families of equilibria that correspond to suboptimal values of the parameters, and as such the learning process may converge to an incorrect distribution. We emphasize that these suboptimal equilibria are not a result of over- or underparametrization by the weights of a neural network, but are inherent in the original parametrization of the approximating distributions. Furthermore, the structure and the stability of these suboptimal equilibria is affected by outliers in the training data.Our goal is to perform a detailed analysis of all the equilibria, their stability, basins of attraction, and the structure of their stable manifolds. We will use this knowledge to understand how one can modify the learning process in such a way that the parameters converge to the correct equilibrium and thus represent the correct ground truth distribution. In particular, our research programme will include the analysis of the dynamics corresponding to single- and multicomponent mixture distributions in the presence of outliers in the training data set. We especially aim at obtaining proper correction formulas for the variance in the presence of outliers in the training data set, which will be derived via a rigorous analysis of the attractors of the emerging dynamical systems.Although our methods heavily rely upon the theory of differential equations and dynamical systems, the anticipated results will be relevant to modern fields of stochastics, machine learning, and artificial intelligence. We believe that they will not only elucidate the pitfalls of learning probability distributions with neural networks, but also help to make the learning process more efficient.

我们的概念验证项目旨在建立机器学习和动力系统理论之间的桥梁。更具体地说，我们开发了一个动态系统范式来学习观测数据的概率分布。近年来，由于人工神经网络能够以参数方式表示概率分布，并有效地从观测数据中学习其参数，因此它已经得到了大量的应用。在我们的项目中，我们研究概率分布的参数（神经网络的输出）在学习过程中是如何演变的。通过考虑数据点数量趋于无穷时的极限，我们可以用一个可处理的微分方程组来描述学习分布参数的过程，我们将详细分析这个过程。正如我们在下面解释的那样，这些动态系统通常具有与参数的次优值相对应的均衡族，因此学习过程可能收敛到不正确的分布。我们强调，这些次优平衡不是神经网络权重参数化过度或不足的结果，而是近似分布的原始参数化所固有的。此外，这些次优平衡点的结构和稳定性受到训练数据中的异常值的影响。我们的目标是对所有的平衡、它们的稳定性、吸引力盆地和它们的稳定流形的结构进行详细的分析。我们将利用这些知识来理解如何修改学习过程，使参数收敛到正确的平衡状态，从而表示正确的基础真值分布。特别是，我们的研究计划将包括在训练数据集中存在异常值时对应的单组分和多组分混合分布的动力学分析。我们特别致力于获得训练数据集中存在异常值时方差的适当校正公式，这将通过对新兴动力系统的吸引子的严格分析得出。虽然我们的方法在很大程度上依赖于微分方程和动力系统理论，但预期的结果将与现代随机学、机器学习和人工智能领域相关。我们相信，它们不仅将阐明用神经网络学习概率分布的陷阱，而且有助于使学习过程更有效。