FreeML: Engineering Networked Machine Learning via Meta-Free Energy Minimisation

FreeML：通过无元能量最小化进行工程网络机器学习

• 批准号: EP/W024101/1
• 负责人: Osvaldo Simeone
• 金额: $ 135.28万
• 依托单位: King's College London
• 依托单位国家: 英国
• 项目类别: Fellowship
• 财政年份: 2023
• 资助国家: 英国
• 起止时间: 2023 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FW024101%2F1
• 关键词: FreeML; Engineering; Networked; Machine; Learning

Inspired by neuroscience, informed by information-theoretic principles, and motivated by modern wireless systems architectures integrating artificial intelligence (AI) and communications, this Fellowship sets out to develop a paradigm-shifting framework for networked machine learning (ML) that is centred on the following ideas.1. Free energy minimisation: According to the free energy principle, agents optimise internal models so as to minimise their information-theoretic surprise vis-a-vis the available data and prior information. This principle offers a basis to reason about epistemic uncertainty ("know when you don't know") in AI agents that is grounded in information-theoretic analyses of out-of-sample generalisation - away from the current narrow focus on point-wise accuracy, towards uncertainty quantification and calibration. A well-calibrated agent can make informed decisions about when to refrain from acting, about when and how to collect or request more data from the environment or other agents, and about how to guard against anomalies or malicious agents.2. Networked meta-learning: In meta-learning, agents do not share an ML model in full as in conventional, centralised, solutions. Rather, only a meta-model is shared as a means to transfer knowledge across agents, while enabling the optimisation of personalised local models. As advocated by FreeML, meta-models can naturally implement the engineering principle of modularity by encompassing a common repository of functions that can be combined to suit the cognitive needs of each agent. This framework bridges the gap between the dominant centralised or joint learning approaches - including also federated learning - and the individual learning baseline, by means of limited model sharing, while still enabling meaningful cooperation with a controlled privacy loss.3. Native integration of wireless communication and learning: Conventional wireless systems are based on the principle of separation between computing and communications. In contrast, the native integration of communications and learning advocated by FreeML embeds wireless communication primitivesas part of the data generating and processing model. Like state-of-the-art integrated solutions, the proposed approach aims at fully utilizing radio channel capacity by avoiding inefficiencies due to separate processing. Unlike existing methods, however, the FreeML framework moves away from the standard problem of communicating under uncertainty (on the communication channel) to the novel problem of communicating uncertainty (on thesolution of the cognitive task) under uncertainty (on the communication channel) in order to support networked meta-learning.Overall, FreeML sets out to study a novel, theoretically principled, paradigm for ML that moves away from the current centralised, accuracy-focused, state of the art in ML to embrace decentralization via wireless connectivity, uncertainty quantification, personalisation, modularity, privacy preservation, and the right to erasure.FreeML will involve three industrial partners -- Intel, InterDigital, and Samsung AI -- that will provide guidance and feedback on aspects related to implementation efficiency, communications, and integration with wireless networks, respectively.This Fellowship proposal builds on the PI's unique inter-disciplinary expertise in information theory, ML, and communications, and is intended to enable a step change in the applicant's career towards a leadership position at the intersection of the fields of engineering and ML/AI. Through this programme, the PI will reach out to a diverse community of STEM students, public, regulators, journalists, and academic colleagues across the two fields to advocate for the central role of engineering for reliable and sustainable ML/AI.

受神经科学的启发，受信息论原理的启发，受集成人工智能(AI)和通信的现代无线系统架构的激励，该奖学金致力于开发一个以以下理念为中心的网络机器学习(ML)范式转换框架。自由能最小化：根据自由能原理，代理人优化内部模型，以最大限度地减少他们对现有数据和先验信息的信息论惊喜。这一原则为人工智能代理中的认知不确定性(“当你不知道时知道”)提供了推理的基础，这种不确定性基于对样本外概括的信息论分析--从目前狭隘地关注逐点精度，转向不确定性量化和校准。经过良好校准的代理可以做出明智的决定，决定何时不采取行动、何时以及如何从环境或其他代理收集或请求更多数据，以及如何防范异常或恶意代理。网络元学习：在元学习中，代理不像在传统的集中式解决方案中那样完全共享ML模型。相反，只有元模型被共享，作为一种在代理之间传递知识的手段，同时允许优化个性化的本地模型。正如FreeML所倡导的那样，元模型可以通过包含一个公共的功能存储库来自然地实现模块化的工程原则，这些功能可以组合在一起以适应每个代理的认知需求。这一框架通过有限的模型共享，在控制隐私损失的情况下，仍然能够进行有意义的合作，从而弥合了主要的集中式或联合学习方法--也包括联合学习--与个人学习基线之间的差距。无线通信和学习的本地集成：传统的无线系统基于计算和通信分离的原则。相比之下，FreeML倡导的通信和学习的本地集成将无线通信原语嵌入到数据生成和处理模型中。与最先进的集成解决方案一样，建议的方法旨在通过避免由于单独处理而导致的低效率来充分利用无线信道容量。然而，与现有方法不同，FreeML框架从(在通信通道上)不确定情况下的通信的标准问题转移到(在通信通道上)不确定情况下的通信不确定性(关于认知任务的解决方案)的新问题，以支持网络化的元学习。总的来说，FreeML着手研究一种新颖的、理论上原则性的ML范例，该范例从目前ML中集中的、以精度为中心的最新技术状态转变为通过无线连接、不确定性量化、个性化、模块化、隐私保护和擦除权来实现去中心化。Free ML将涉及三个行业合作伙伴--英特尔、InterDigital、这项奖学金建议建立在PI在信息理论、ML和通信方面的独特跨学科专业知识的基础上，旨在使申请者的职业生涯朝着工程和ML/AI领域的领导职位迈进一步。通过这项计划，PI将接触到STEM学生、公众、监管者、记者和两个领域的学术同行的不同社区，倡导工程在可靠和可持续的ML/AI中发挥核心作用。

项目成果

Neuromorphic Wireless Cognition: Event-Driven Semantic Communications for Remote Inference

• DOI: 10.1109/tccn.2023.3236940
• 发表时间: 2022-06
• 期刊: IEEE Transactions on Cognitive Communications and Networking
• 影响因子: 8.6
• 作者: Jiechen Chen;N. Skatchkovsky;O. Simeone

Calibrating AI Models for Few-Shot Demodulation VIA Conformal Prediction

• DOI: 10.1109/icassp49357.2023.10096780
• 发表时间: 2022-10
• 期刊: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• 作者: K. Cohen;Sangwoo Park;O. Simeone;S. Shamai

Guaranteed Dynamic Scheduling of Ultra-Reliable Low-Latency Traffic via Conformal Prediction

• DOI: 10.1109/lsp.2023.3264939
• 发表时间: 2023-02
• 期刊: IEEE Signal Processing Letters
• 影响因子: 3.9
• 作者: K. Cohen;Sangwoo Park;O. Simeone;P. Popovski;S. Shamai

Calibration-Aware Bayesian Learning

• DOI: 10.1109/mlsp55844.2023.10285894
• 发表时间: 2023-05
• 期刊: 2023 IEEE 33rd International Workshop on Machine Learning for Signal Processing (MLSP)
• 作者: Jiayi Huang;Sangwoo Park;O. Simeone

Calibrating AI Models for Wireless Communications via Conformal Prediction

通过共形预测校准无线通信人工智能模型

• DOI: 10.1109/tmlcn.2023.3319282
• 发表时间: 2023
• 期刊: IEEE Transactions on Machine Learning in Communications and Networking
• 作者: Cohen K