Deepcode: Feedback Codes via Deep Learning
Deepcode: Feedback Codes via Deep Learning
复制标题
DOI：
10.1109/jsait.2020.2986752
复制
发表时间：
2018-07
期刊：
影响因子：
--
通讯作者：
Hyeji Kim;Yihan Jiang;Sreeram Kannan;Sewoong Oh;P. Viswanath
中科院分区：
文献类型：
--
作者：
Hyeji Kim;Yihan Jiang;Sreeram Kannan;Sewoong Oh;P. Viswanath
在统计上定义良好的通道上可靠地通信的代码设计是一项重要的努力，涉及深度数学研究和广泛的实用应用。在这项工作中，我们介绍了通过深度学习获得的第一个代码家族，该家族在数十年的研究中大大优于最先进的代码。所考虑的通信渠道是带有反馈的高斯噪声通道，其研究是由香农发起的。从理论上讲，反馈是为了提高沟通的可靠性，但没有成功构建的实际代码。我们通过将信息理论洞察力与基于重复的神经网络的编码器和解码器和谐地整合在一起，以创建新颖的代码，从而超过3个数量级的可靠性，并以SNR获得3DB增益。我们还证明了代码的几种理想属性：（a）对较大的块长度的概括，（b）与已知代码的合成性，以及（c）适应实际约束。该结果对编码理论也具有更广泛的影响：即使频道具有清晰的数学模型，深度学习方法，当与渠道特定信息理论理论见解结合使用时，也可能会超过数十年来构建的最先进的代码数学研究。
The design of codes for communicating reliably over a statistically well defined channel is an important endeavor involving deep mathematical research and wide-ranging practical applications. In this work, we present the first family of codes obtained via deep learning, which significantly outperforms state-of-the-art codes designed over several decades of research. The communication channel under consideration is the Gaussian noise channel with feedback, whose study was initiated by Shannon; feedback is known theoretically to improve reliability of communication, but no practical codes that do so have ever been successfully constructed. We break this logjam by integrating information theoretic insights harmoniously with recurrent-neural-network based encoders and decoders to create novel codes that outperform known codes by 3 orders of magnitude in reliability and achieve a 3dB gain in terms of SNR. We also demonstrate several desirable properties of the codes: (a) generalization to larger block lengths, (b) composability with known codes, and (c) adaptation to practical constraints. This result also has broader ramifications for coding theory: even when the channel has a clear mathematical model, deep learning methodologies, when combined with channel-specific information-theoretic insights, can potentially beat state-of-the-art codes constructed over decades of mathematical research.