Generalization Bounds for Neural Belief Propagation Decoders
Generalization Bounds for Neural Belief Propagation Decoders
复制标题
DOI：
10.1109/isit54713.2023.10206901
复制
发表时间：
2023-05
期刊：
影响因子：
--
通讯作者：
S. Adiga;Xin Xiao;Ravi Tandon;Bane V. Vasic;Tamal Bose
中科院分区：
文献类型：
--
作者：
S. Adiga;Xin Xiao;Ravi Tandon;Bane V. Vasic;Tamal Bose
基于机器学习的方法越来越多地用于设计下一代通信系统的解码器。其中一个广泛使用的框架是神经信念传播（NBP），它将信念传播（BP）迭代展开成一个深度神经网络，并以数据驱动的方式训练参数。NBP解码器已被证明是对经典解码算法的改进。本文研究了NBP解码器的泛化能力。具体来说，解码器的泛化差距是经验误码率和期望误码率之间的差异。我们提出了新的理论结果，它约束了这一差距，并显示了对解码器复杂性的依赖，在代码参数（块长度、消息长度、变量/检查节点度）、解码迭代和训练数据集大小方面。给出了正则和不规则奇偶校验矩阵的结果。据我们所知，这是第一组关于基于神经网络的解码器泛化性能的理论结果。我们给出了实验结果，显示了泛化差距与训练数据集大小的依赖关系，以及不同编码的解码迭代。
Machine learning based approaches are being increasingly used for designing decoders for next generation communication systems. One widely used framework is neural belief propagation (NBP), which unfolds the belief propagation (BP) iterations into a deep neural network and the parameters are trained in a data-driven manner. NBP decoders have been shown to improve upon classical decoding algorithms. In this paper, we investigate the generalization capabilities of NBP decoders. Specifically, the generalization gap of a decoder is the difference between empirical and expected bit-error-rate(s). We present new theoretical results which bound this gap and show the dependence on the decoder complexity, in terms of code parameters (blocklength, message length, variable/check node degrees), decoding iterations, and the training dataset size. Results are presented for both regular and irregular parity-check matrices. To the best of our knowledge, this is the first set of theoretical results on generalization performance of neural network based decoders. We present experimental results to show the dependence of generalization gap on the training dataset size, and decoding iterations for different codes.