CAREER: An Information-Theoretic Approach to Communication-Constrained Statistical Learning

该项目旨在开发一种信息论方法来解决通信受限的统计学习问题，该问题涉及位于大型网络节点的多个学习代理。这种方法将建立在最近引入的网络信息理论中的协调范例上，该范例着眼于通信资源的最佳使用方面的多终端问题，以便在网络的节点之间建立某些期望的统计相关性。主要的理论目标是明确确定带宽限制、丢失、延迟和缺乏中央协调对网络上统计学习算法性能的影响。该项目将系统地探索多终端环境中学习的基本限制，并设计高效、可实施和健壮的编码/解码方案。在该项目下开发的理论将是来自机器学习的概率技术(如经验过程理论)和多终端信息理论(如分布式有损信源编码)的新综合。作为更广泛的影响，该项目将为机器学习在智能电网、医疗保健信息学、交通网络和网络安全等领域的大规模、分布式应用提供关键使能技术。在存在重大模型不确定性的情况下，统计机器学习正在成为根据经验观察做出准确预测的主导范式。然而，这一领域的大多数研究活动都是在脱离复杂网络的现实和信息传输和处理的所有随之而来的限制的情况下进行的：人们经常假设学习所需的数据可以立即获得，精度任意，并且在单一位置。然而，考虑到提供给机器学习算法的大多数数据是越来越多地在大规模网络上生成、交换、存储和处理的事实，迫切需要摒弃这一假设，从而考虑网络效应。作为该项目的一部分开发的理论和算法将确保通过网络将相关数据提供给正确的决策者，同时确保根据收到的信息做出准确的决定。该项目的研究部分与教育和外联计划紧密结合，包括开发和教授专门针对工科学生的机器学习新课程。

This project aims to develop an information-theoretic approach to communication-constrained statistical learning problems involving multiple learning agents located at the nodes of a large network. This approach will build on the recently introduced coordination paradigm within network information theory, which looks at multiterminal problems in terms of optimal use of communication resources in order to establish some desired statistical correlations between the nodes of a network. The main theoretical goal is to explicitly identify the effect of bandwidth limitations, losses, delays, and lack of central coordination on the performance of statistical learning algorithms over networks. The project will systematically explore the fundamental limits of learning in multiterminal settings and design efficiently implementable and robust coding/decoding schemes. The theory developed under this project will be a novel synthesis of probabilistic techniques from machine learning (such as empirical process theory) and of multiterminal information theory (such as distributed lossy source coding).As a broader impact, this project will provide key enabling technologies for large-scale, distributed applications of machine learning in such domains as smart grids, health-care informatics, transportation networks, and cybersecurity. Statistical machine learning is emerging as a dominant paradigm for making accurate predictions on the basis of empirical observations in the presence of significant model uncertainty. Most of the research activity in this field, however, has taken place in isolation from the realities of complex networks and all the attendant limitations on information transmission and processing: it is frequently assumed that the data needed for learning are available instantly, with arbitrary precision, and at a single location. However, given the fact that most data fed to machine learning algorithms are increasingly generated, exchanged, stored and processed over large-scale networks, there is a pressing need to dispense with this assumption and thus take network effects into consideration. The theory and the algorithms developed as part of this project will ensure that the relevant data are delivered over the network to the right decision-makers, while securing accurate decisions made on the basis of the received information. The research component of the project is tightly integrated with an education and outreach plan, including development and teaching of new courses on machine learning aimed specifically at engineering students.