The Geometry of Deep Networks: Power Diagram Subdivision
The Geometry of Deep Networks: Power Diagram Subdivision
复制标题
DOI：
--
复制
发表时间：
2019-05
期刊：
影响因子：
--
通讯作者：
Randall Balestriero;Romain Cosentino;B. Aazhang;Richard Baraniuk
中科院分区：
文献类型：
--
作者：
Randall Balestriero;Romain Cosentino;B. Aazhang;Richard Baraniuk
我们研究具有分段仿射和凸非线性的深度（神经）网络（DN）的几何形状。此类 DN 的层已被证明是{\em最大仿射样条运算符}（MASO），它们划分输入空间并将区域相关的仿射映射应用于其输入以产生输出。我们证明每个 MASO 层的输入空间划分对应于一个{\em 功率图}（经典 Voronoi 平铺的扩展），其中许多区域相对于单元（神经元）的数量呈指数增长。我们进一步表明，MASO 层的组合（例如整个 DN）会产生逐步细分的功率图并提供其分析形式。细分过程将仿射图限制在（指数级多）功率图区域上，以大大降低其复杂性。对于分类问题，我们获得输入空间中 MASO DN 决策边界的公式以及取决于 DN 非线性、权重和架构的曲率度量。大量的数值实验支持并扩展了我们的理论结果。
We study the geometry of deep (neural) networks (DNs) with piecewise affine and convex nonlinearities. The layers of such DNs have been shown to be {\em max-affine spline operators} (MASOs) that partition their input space and apply a region-dependent affine mapping to their input to produce their output. We demonstrate that each MASO layer's input space partitioning corresponds to a {\em power diagram} (an extension of the classical Voronoi tiling) with a number of regions that grows exponentially with respect to the number of units (neurons). We further show that a composition of MASO layers (e.g., the entire DN) produces a progressively subdivided power diagram and provide its analytical form. The subdivision process constrains the affine maps on the (exponentially many) power diagram regions to greatly reduce their complexity. For classification problems, we obtain a formula for a MASO DN's decision boundary in the input space plus a measure of its curvature that depends on the DN's nonlinearities, weights, and architecture. Numerous numerical experiments support and extend our theoretical results.