Principles of Riemannian Geometry in Neural Networks
Principles of Riemannian Geometry in Neural Networks
复制标题
神经网络中的黎曼几何原理
DOI：
--
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
A. Ray
中科院分区：
文献类型：
--
作者：
Michael Hauser;A. Ray
本研究涉及神经网络的意义上的几何变换作用于坐标表示的基础数据流形的数据是从采样。它是试图在黎曼几何的背景下构建一个形式化的神经网络一般理论的一部分。从这个角度来看，下面的理论结果是开发和证明前馈网络。首先，它表明，残差神经网络是有限差分近似的动力系统的一阶微分方程，而不是普通的网络是静态的。这意味着网络正在学习控制代表数据的坐标变换的微分方程系统。其次，它表明，一个封闭形式的解决方案的度量张量的基础数据流形上可以找到通过反向传播的坐标表示学习的神经网络本身。这是制定在一个正式的抽象意义上作为一个序列的李群行动的度量纤维空间中的主要和相关的丛的数据流形。玩具实验是为了证实所提出的理论的一部分，并提供关于神经网络如何对数据进行操作的直观信息。
This study deals with neural networks in the sense of geometric transformations acting on the coordinate representation of the underlying data manifold which the data is sampled from. It forms part of an attempt to construct a formalized general theory of neural networks in the setting of Riemannian geometry. From this perspective, the following theoretical results are developed and proven for feedforward networks. First it is shown that residual neural networks are finite difference approximations to dynamical systems of first order differential equations, as opposed to ordinary networks that are static. This implies that the network is learning systems of differential equations governing the coordinate transformations that represent the data. Second it is shown that a closed form solution of the metric tensor on the underlying data manifold can be found by backpropagating the coordinate representations learned by the neural network itself. This is formulated in a formal abstract sense as a sequence of Lie group actions on the metric fibre space in the principal and associated bundles on the data manifold. Toy experiments were run to confirm parts of the proposed theory, as well as to provide intuitions as to how neural networks operate on data.