CAREER: An Information-Theoretic Approach to Communication-Constrained Statistical Learning

职业：通信受限统计学习的信息论方法

• 批准号: 1254041
• 负责人: Maxim Raginsky
• 金额: $ 51.84万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-02-01 至 2022-01-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1254041&HistoricalAwards=false
• 关键词: CAREER; Information; Theoretic; Approach; Communication

This project aims to develop an information-theoretic approach to communication-constrained statistical learning problems involving multiple learning agents located at the nodes of a large network. This approach will build on the recently introduced coordination paradigm within network information theory, which looks at multiterminal problems in terms of optimal use of communication resources in order to establish some desired statistical correlations between the nodes of a network. The main theoretical goal is to explicitly identify the effect of bandwidth limitations, losses, delays, and lack of central coordination on the performance of statistical learning algorithms over networks. The project will systematically explore the fundamental limits of learning in multiterminal settings and design efficiently implementable and robust coding/decoding schemes. The theory developed under this project will be a novel synthesis of probabilistic techniques from machine learning (such as empirical process theory) and of multiterminal information theory (such as distributed lossy source coding).As a broader impact, this project will provide key enabling technologies for large-scale, distributed applications of machine learning in such domains as smart grids, health-care informatics, transportation networks, and cybersecurity. Statistical machine learning is emerging as a dominant paradigm for making accurate predictions on the basis of empirical observations in the presence of significant model uncertainty. Most of the research activity in this field, however, has taken place in isolation from the realities of complex networks and all the attendant limitations on information transmission and processing: it is frequently assumed that the data needed for learning are available instantly, with arbitrary precision, and at a single location. However, given the fact that most data fed to machine learning algorithms are increasingly generated, exchanged, stored and processed over large-scale networks, there is a pressing need to dispense with this assumption and thus take network effects into consideration. The theory and the algorithms developed as part of this project will ensure that the relevant data are delivered over the network to the right decision-makers, while securing accurate decisions made on the basis of the received information. The research component of the project is tightly integrated with an education and outreach plan, including development and teaching of new courses on machine learning aimed specifically at engineering students.

该项目旨在开发一种信息理论方法，以涉及涉及大型网络节点的多个学习代理的统计学习问题。这种方法将基于网络信息理论中最近引入的协调范式，该范式从最佳使用通信资源来研究多端问题，以建立网络节点之间的一些理想的统计相关性。主要的理论目标是明确确定带宽限制，损失，延迟和中央协调对网络统计学习算法的性能的影响。该项目将系统地探索多端设置中学习的基本局限性，并有效地实施和可靠的编码/解码方案。该项目下开发的理论将是机器学习（例如经验过程理论）和多播信息理论（例如分布式有损耗的源编码）的新型概念。作为更广泛的影响，该项目将为大规模的启用技术提供重要的启用技术，用于大规模的，诸如智能网络的智能网络，医疗服务，cy cy and cy and chetersick and Cyersick and Cyersick and Cyersicksick and Cyersicksick and Cyersicksicksick and Cyersicksicksick and cy cy，，cy cy and cy and cy sytsick and cy sytsick and cy and cy，，cy sy。统计机器学习正在作为主要模型不确定性的经验观察中进行准确预测的主要范式。但是，该领域的大多数研究活动都是与复杂网络的现实以及所有随之而来的有关信息传输和处理的限制是孤立的：经常假定，学习所需的数据可以立即获得，具有任意精度，并且在一个单个位置。但是，鉴于大多数馈送到机器学习算法的数据越来越多地生成，交换，存储和处理在大规模网络上，因此有迫切需要消除此假设，从而考虑到网络效应。作为该项目的一部分而开发的理论和算法将确保通过网络将相关数据传递给正确的决策者，同时确保基于收到的信息确保准确的决策。该项目的研究组成部分与教育和外展计划紧密整合，包括针对工程专业学生的机器学习的新课程的开发和教学。