FreeML: Engineering Networked Machine Learning via Meta-Free Energy Minimisation

受神经科学的启发，信息理论原理的启发，并受到集成人工智能（AI）和通信的现代无线系统架构的激励，该奖学金旨在为网络机器学习（ML）开发一个范式转换框架，该框架以以下想法为中心。自由能最小化：根据自由能原理，代理优化内部模型，以最大限度地减少其信息理论的惊喜，维斯于现有的数据和先验信息。这一原则为人工智能主体中的认知不确定性（“当你不知道时就知道”）提供了一个推理基础，它基于对样本外泛化的信息理论分析-远离当前对逐点准确性的狭隘关注，走向不确定性量化和校准。一个经过良好校准的代理可以做出明智的决定，何时避免采取行动，何时以及如何从环境或其他代理收集或请求更多数据，以及如何防范异常或恶意代理。网络元学习：在元学习中，代理不像传统的集中式解决方案那样完全共享ML模型。相反，只有一个元模型被共享，作为跨代理传递知识的一种手段，同时能够优化个性化的本地模型。正如FreeML所倡导的那样，元模型可以通过包含一个通用的功能库来自然地实现模块化的工程原则，这些功能库可以组合起来以满足每个代理的认知需求。该框架通过有限的模型共享弥合了主导的集中式或联合学习方法（包括联邦学习）与个人学习基线之间的差距，同时仍然能够在可控的隐私损失下进行有意义的合作。无线通信和学习的原生集成：传统的无线系统基于计算和通信分离的原则。相反，FreeML倡导的通信和学习的本地集成将无线通信嵌入数据生成和处理模型的一部分。像国家的最先进的综合解决方案，所提出的方法旨在充分利用无线电信道容量，避免效率低下，由于单独的处理。然而，与现有方法不同，FreeML框架摆脱了在不确定性下进行通信的标准问题（关于沟通渠道）沟通不确定性的新问题（关于认知任务的解决）在不确定性下（在通信信道上），以支持网络元学习。总的来说，FreeML开始研究一种新颖的，理论上有原则的，FreeML是一种机器学习的范式，它摆脱了当前集中的、以准确性为中心的机器学习技术，通过无线连接、不确定性量化、个性化、模块化、隐私保护和擦除权来拥抱去中心化。FreeML将涉及三个工业合作伙伴--英特尔、InterDigital、和三星人工智能-将分别就实施效率，通信和与无线网络集成的相关方面提供指导和反馈。该奖学金提案建立在PI在信息论，ML和通信方面独特的跨学科专业知识基础上，旨在使申请人的职业生涯发生重大变化，在工程和ML/AI领域的交叉点上担任领导职务。通过该计划，PI将接触到STEM学生、公众、监管机构、记者和学术界的多元化社区，倡导工程在可靠和可持续的ML/AI中发挥核心作用。

Inspired by neuroscience, informed by information-theoretic principles, and motivated by modern wireless systems architectures integrating artificial intelligence (AI) and communications, this Fellowship sets out to develop a paradigm-shifting framework for networked machine learning (ML) that is centred on the following ideas.1. Free energy minimisation: According to the free energy principle, agents optimise internal models so as to minimise their information-theoretic surprise vis-a-vis the available data and prior information. This principle offers a basis to reason about epistemic uncertainty ("know when you don't know") in AI agents that is grounded in information-theoretic analyses of out-of-sample generalisation - away from the current narrow focus on point-wise accuracy, towards uncertainty quantification and calibration. A well-calibrated agent can make informed decisions about when to refrain from acting, about when and how to collect or request more data from the environment or other agents, and about how to guard against anomalies or malicious agents.2. Networked meta-learning: In meta-learning, agents do not share an ML model in full as in conventional, centralised, solutions. Rather, only a meta-model is shared as a means to transfer knowledge across agents, while enabling the optimisation of personalised local models. As advocated by FreeML, meta-models can naturally implement the engineering principle of modularity by encompassing a common repository of functions that can be combined to suit the cognitive needs of each agent. This framework bridges the gap between the dominant centralised or joint learning approaches - including also federated learning - and the individual learning baseline, by means of limited model sharing, while still enabling meaningful cooperation with a controlled privacy loss.3. Native integration of wireless communication and learning: Conventional wireless systems are based on the principle of separation between computing and communications. In contrast, the native integration of communications and learning advocated by FreeML embeds wireless communication primitivesas part of the data generating and processing model. Like state-of-the-art integrated solutions, the proposed approach aims at fully utilizing radio channel capacity by avoiding inefficiencies due to separate processing. Unlike existing methods, however, the FreeML framework moves away from the standard problem of communicating under uncertainty (on the communication channel) to the novel problem of communicating uncertainty (on thesolution of the cognitive task) under uncertainty (on the communication channel) in order to support networked meta-learning.Overall, FreeML sets out to study a novel, theoretically principled, paradigm for ML that moves away from the current centralised, accuracy-focused, state of the art in ML to embrace decentralization via wireless connectivity, uncertainty quantification, personalisation, modularity, privacy preservation, and the right to erasure.FreeML will involve three industrial partners -- Intel, InterDigital, and Samsung AI -- that will provide guidance and feedback on aspects related to implementation efficiency, communications, and integration with wireless networks, respectively.This Fellowship proposal builds on the PI's unique inter-disciplinary expertise in information theory, ML, and communications, and is intended to enable a step change in the applicant's career towards a leadership position at the intersection of the fields of engineering and ML/AI. Through this programme, the PI will reach out to a diverse community of STEM students, public, regulators, journalists, and academic colleagues across the two fields to advocate for the central role of engineering for reliable and sustainable ML/AI.