CIF: Small: MoDL: Interpreting Deep-Learned Error-Correcting Codes

信道编码纠错码的理论和发展是一个有着悠久而辉煌历史的领域。通过数学和算法的突破，在开发接近容量的代码方面取得了巨大进展。强大的信道码对于确保我们的互联社会及其技术创新的可靠通信至关重要。最近，在纠错码的开发中出现了一种新的范式：使用机器学习以端到端方式学习编码器或解码器。虽然在编码理论中超越几十年的工作并不容易，但其中一些代码在实际块长度上表现出色，有些代码能够在提供鲁棒性优势的同时实现类似的性能，有些代码能够在具有反馈的通道中显着优于现有代码。这些进步是更大图景的一部分：机器学习在通信堆栈的各个部分，特别是在无线领域，发挥着越来越重要的作用。虽然深度学习代码是由特定的训练有素的神经网络明确给出的（通常具有非常大量的训练参数），但它们一直受到怀疑，其中一些至少源于这些深度学习的纠错代码被视为“黑盒子”，迄今为止除了有限的实验之外，几乎没有证明它们何时或为什么表现良好。我们的研究将帮助工程师理解、解释和信任深度学习纠错代码的性能。这个项目将最终帮助智能和仔细地将它们整合到广泛的理论研究和实践系统中。此外，它将通过提供工具和算法来促进深度学习在其他科学领域的应用，这些工具和算法将使新的机器学习工件能够被解释并与现有的形式主义相关联。培养下一代数据科学工程师批判性地解释和理解他们的系统所学习的东西是很重要的：拟议的研究将从一个新的、严格的角度来做到这一点，这是由纠错码的定量性质所支持的，具有精确的度量和准确的分布和统计建模。我们提出了一个变革性的研究项目，该项目建立在有希望的初步工作的基础上，将为以结构化、正式的方式解释深度学习的纠错代码奠定基础。这个跨学科的工作线融合了编码和信息论，可解释的人工智能，深度学习，计算学习理论，优化，系统识别和布尔函数分析的各个方面，以实现以下雄心勃勃的目标：1)开发用于解释深度学习纠错码的算法工具，2)使用这些工具来理解深度学习纠错码，3)根据实验结果提供深度学习纠错码的形式化表征，4)使用它来设计改进的深度学习纠错码，以及5)使用见解来改进传统代码。所获得的结果预计将为一般的可解释人工智能提供重要的新工具，并将使我们能够将研究计划扩展到网络和联合源信道编码设置的深度学习代码。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The theory and development of error-correcting codes for channel coding is a field with a long and distinguished history. Enormous progress has been made through mathematical and algorithmic breakthroughs toward developing codes that approach capacity. Powerful channel codes are crucial in ensuring reliable communication for our connected society and its technological innovations. Recently, a new paradigm has emerged in the development of error-correcting codes: learn, using machine learning, the encoders or decoders in an end-to-end fashion. While it has not been easy to outperform decades of work in coding theory, some of these codes excel at practical blocklengths, some are able to achieve similar performance while offering robustness benefits, and some are able to dramatically outperform existing codes in channels with feedback. Such advances are part of a larger picture: machine learning is playing an increasingly important role in various pieces of the communication stack, particularly in the wireless domain. While deep-learned codes are explicitly given by the specific trained neural networks (often with a very large number of trained parameters), they have been met with skepticism, some of which at least stems from these deep-learned error-correcting codes being viewed as "black boxes," with little so far beyond limited experiments to demonstrate when or why they perform well. Our research will help engineers understand, interpret, and trust the performance of deep-learned error-correcting codes. This project will ultimately help intelligently and carefully integrate them into the extensive body of theoretical research as well as practical systems. Further, it will advance the use of deep learning in other scientific fields by providing tools and algorithms that will enable the new machine-learned artifacts to be interpreted and related to existing formalisms. It is important to train our next generation of data-science engineers to critically interpret and understand what their systems learn: the proposed research will do so from a new, rigorous angle enabled by the quantitative nature of error-correcting codes, with precise metrics and accurate modeling of distributions and statistics. We present a transformative research program that builds upon promising preliminary work that will lay the foundation for interpreting deep-learned error-correcting codes in a structured, formal manner. This interdisciplinary line of work merges aspects of coding and information theory, explainable artificial intelligence, deep learning, computational learning theory, optimization, system identification, and Boolean function analysis towards the following ambitious goals: 1) developing algorithmic tools for the interpretation of deep-learned error-correcting codes, 2) using these tools to understand deep-learned error-correcting codes, 3) providing a formal characterization of deep-learned error-correcting codes based on the experimental results, 4) using this to design improved deep-learned error-correcting codes, and 5) using the insights to improve traditional codes. Results obtained are expected to provide important new tools for explainable artificial intelligence in general and will allow us to broaden the research program to deep-learned codes for the network and joint source-channel coding settings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.