Convolutional Filters and Neural Networks With Noncommutative Algebras
Convolutional Filters and Neural Networks With Noncommutative Algebras
复制标题
DOI：
10.1109/tsp.2023.3293716
复制
发表时间：
2021-08
影响因子：
5.4
通讯作者：
Alejandro Parada-Mayorga;Alejandro Ribeiro
中科院分区：
文献类型：
--
作者：
Alejandro Parada-Mayorga;Alejandro Ribeiro
本文介绍并研究了非交换卷积神经网络的代数推广。我们利用代数信号处理理论对卷积非交换结构进行建模，并推导出具体的稳定性界，扩展了文献中对交换卷积神经网络的稳定性界。我们证明了非交换卷积结构对算子空间上的变形是稳定的。我们发展了非交换信号模型的频谱表示，以表明非交换滤波器相互独立地处理傅里叶分量。特别地，我们证明了非交换模型中信号的谱分解虽然与维数大于1的特征空间相关，但在稳定性和选择性之间存在权衡，这是由低维矩阵空间中的矩阵多项式函数控制的。这种权衡表明，当代数中的滤波器被限制为稳定时，网络中存在可判别性的损失，该损失由点向非线性补偿。本文得出的结果在非交换卷积结构（如群神经网络、多图神经网络和四元数神经网络）中有直接的应用和意义，为此我们提供了一组数值实验，显示了它们在存在扰动时的行为。
In this paper we introduce and study the algebraic generalization of non commutative convolutional neural networks. We leverage the theory of algebraic signal processing to model convolutional non commutative architectures, and we derive concrete stability bounds that extend those obtained in the literature for commutative convolutional neural networks. We show that non commutative convolutional architectures can be stable to deformations on the space of operators. We develop the spectral representation of non commutative signal models to show that non commutative filters process Fourier components independently of each other. In particular we prove that although the spectral decompositions of signals in non commutative models are associated to eigenspaces of dimension larger than one, there exists a trade-off between stability and selectivity, which is controlled by matrix polynomial functions in spaces of matrices of low dimension. This tradeoff shows how when the filters in the algebra are restricted to be stable, there is a loss in discriminability that is compensated in the network by the pointwise nonlinearities. The results derived in this paper have direct applications and implications in non commutative convolutional architectures such as group neural networks, multigraph neural networks, and quaternion neural networks, for which we provide a set of numerical experiments showing their behavior when perturbations are present.