CAREER: An Information-Theoretic Approach to Communication-Constrained Statistical Learning

该项目旨在开发一种信息理论方法来解决通信受限的统计学习问题，该问题涉及位于大型网络节点处的多个学习代理。这种方法将建立在最近推出的网络信息理论，它着眼于多终端问题的通信资源的最佳利用，以建立一些理想的统计网络的节点之间的相关性的协调范式。主要的理论目标是明确识别带宽限制、损失、延迟和缺乏中央协调对网络上统计学习算法性能的影响。该项目将系统地探索在多终端环境中学习的基本限制，并设计有效实施和强大的编码/解码方案。在这个项目下开发的理论将是一种新的机器学习概率技术的综合（如经验过程理论）和多端信息理论（如分布式有损源编码）。作为一个更广泛的影响，该项目将提供关键的使能技术，用于智能电网，医疗保健信息学，交通网络，和网络安全。统计机器学习正在成为在存在显著模型不确定性的情况下基于经验观察进行准确预测的主导范式。然而，这一领域的大多数研究活动都是在与复杂网络的现实以及随之而来的信息传输和处理方面的所有限制相隔离的情况下进行的：人们经常假设，学习所需的数据可以立即获得，具有任意的精度，并且在一个位置。然而，考虑到大多数输入机器学习算法的数据越来越多地在大规模网络上生成、交换、存储和处理，迫切需要放弃这种假设，从而考虑网络效应。作为该项目的一部分开发的理论和算法将确保相关数据通过网络传递给正确的决策者，同时确保根据收到的信息做出准确的决策。该项目的研究部分与教育和推广计划紧密结合，包括专门针对工程专业学生的机器学习新课程的开发和教学。

This project aims to develop an information-theoretic approach to communication-constrained statistical learning problems involving multiple learning agents located at the nodes of a large network. This approach will build on the recently introduced coordination paradigm within network information theory, which looks at multiterminal problems in terms of optimal use of communication resources in order to establish some desired statistical correlations between the nodes of a network. The main theoretical goal is to explicitly identify the effect of bandwidth limitations, losses, delays, and lack of central coordination on the performance of statistical learning algorithms over networks. The project will systematically explore the fundamental limits of learning in multiterminal settings and design efficiently implementable and robust coding/decoding schemes. The theory developed under this project will be a novel synthesis of probabilistic techniques from machine learning (such as empirical process theory) and of multiterminal information theory (such as distributed lossy source coding).As a broader impact, this project will provide key enabling technologies for large-scale, distributed applications of machine learning in such domains as smart grids, health-care informatics, transportation networks, and cybersecurity. Statistical machine learning is emerging as a dominant paradigm for making accurate predictions on the basis of empirical observations in the presence of significant model uncertainty. Most of the research activity in this field, however, has taken place in isolation from the realities of complex networks and all the attendant limitations on information transmission and processing: it is frequently assumed that the data needed for learning are available instantly, with arbitrary precision, and at a single location. However, given the fact that most data fed to machine learning algorithms are increasingly generated, exchanged, stored and processed over large-scale networks, there is a pressing need to dispense with this assumption and thus take network effects into consideration. The theory and the algorithms developed as part of this project will ensure that the relevant data are delivered over the network to the right decision-makers, while securing accurate decisions made on the basis of the received information. The research component of the project is tightly integrated with an education and outreach plan, including development and teaching of new courses on machine learning aimed specifically at engineering students.