Wasserstein-Type Gradient Flow via Propagation by Chaos for a Continuous Formulation of a Shallow Neural Network

在过去的几十年里，统计学、数据分析和人工智能等科学学科的进步和普及突然激增。这种增长可以归因于机器学习优化框架的发现，这是一种简单，可扩展和可参数化的数据插值方法。它的应用涵盖了广泛的行业领域，包括金融，计算机编程，3D图形，数据分析，医疗保健，流行病学，法律，城市研究，动物行为等等。尽管几乎每个人都意识到这些算法在我们的日常生活中变得越来越相互交织和无处不在，但与直觉相反，我们还没有开发出一个令人满意的数学模型来描述这些机器的行为，并且仍处于研究的早期阶段。这反映在我们称这些算法为“黑箱”的事实上。因此，我们必须研究机器学习框架和梯度下降算法。为了优化这些机器的性能和效率，我们必须了解它们的内部工作方式。这将使我们能够调整这些机器，以产生更好的结果，更少的计算资源和data.We专注于近似数据集与单层神经网络的均方误差和一些正则化的梯度下降算法的装置的具体问题。单层神经网络由有限个节点组成，每个节点由一个参数和一个权重参数化。今天，最广泛使用的数学框架来分析这个问题之一是采用单层神经网络的平均场描述，并调查关于平方Wasserstein度量的梯度流的适定性和性质。然而，这种表示包含冗余，在这个意义上，神经网络对节点权重的方差是不变的。有很多证据表明，在梯度下降算法中，节点权重的方差将单调下降。特别是一个初始的零方差的权重参数将在整个梯度流中传播，也就是说，我们最终与所谓的Young measures的曲线。本项目的目的是建立一个严格的数学证明所述猜想。此外，这个新的框架可能会为我们提供缺失的工具，以确定定量收敛界或梯度下降算法的渐近极限的属性，这两者仍然是知之甚少。此外，这个框架给了我们一个独特的机会，也分析的属性的演变和规律性的单层神经网络的权重表示为一个函数的parameters.The项目是新颖的，在这个意义上说，没有已知的来源，研究几何形状的杨措施相对于梯度下降方程。此外，这项研究将给我们更多的洞察力的几何平均场解的梯度流方程。

Scientific disciplines such as statistics, data analysis and artificial intelligence have witnessed a sudden surge of progress and popularity during the last few decades. This growth can be attributed to the discovery of the machine learning optimization framework, which is a simple, scalable and parameterizable method of interpolating data. Its applications span a wide area of industry including finance, computer programming, 3D graphics, data analysis, health care, epidemiology, law, urban study, animal behaviour and many more. Even though almost everyone is aware that these algorithms are becoming more and more intertwined and ubiquitous in our daily life, rather counterintuitively, we have yet to develop a satisfactory mathematical model to describe the behaviour of these machines and are still at the early stages of research. This is reflected in the fact we call these algorithms "black boxes".It is therefore imperative that we research the machine learning framework and the gradient descent algorithm. To optimize the performance and efficiency of these machines, we must gain an understanding of how they work internally. This will allow us to adjust these machines to produce better results with less computational resources and data.We focus on the specific problem of approximating a dataset with a single layer neural network with respect to the mean square error and some regularization by means of the gradient descent algorithm. A single layer neural network is comprised of finite nodes, each parameterised by a parameter and a weight. Today, one of the most widely used mathematical frameworks to analyse this problem is to employ a mean-field description of the single layer neural network and to investigate the well-posedness and properties of the gradient flow with respect to the square Wasserstein metric. However, this representation contains redundancy, in the sense that the neural network is invariant to the variance of the weight of a node. There is much evidence that suggests that the variance in the weight of a node will monotonically decrease during the gradient descent algorithm. In particular an initial zero variance in the weight parameter will be propagated throughout the entire gradient flow, that is we end up with a curve of so-called Young measures.The aim of this project is to establish a rigorous mathematical justification to prove the stated conjecture. Moreover, this new framework may provide us with the missing tools to determine quantitative convergence bounds or properties of the asymptotic limit of the gradient descent algorithm, both of which are still poorly understood. Additionally, this framework gives us the unique opportunity to also analyse the properties in evolution and regularity of the weights of the single layer neural network expressed as a function of the parameters.The project is novel, in the sense that there are no known sources that study the geometry of Young measures with respect to the gradient descent equation. Furthermore, this research will give us more insight in the geometry of mean-field solutions to the gradient flow equation.