EAGER:Real Time Federated Learning using Kernel Methods

我们越来越多地看到越来越多的动态数据来自各种不同的应用程序，从电网到交通系统，从医疗网络到社交网络数据。这些数据来自各种来源，包括传感器网络和物联网应用。一个共同的特点是，这些数据中的大部分是分布的（在边缘），需要建立方法来处理、学习和根据这些数据做出决策。一种方法是在中央处理器（在云中）收集所有数据，并在计算资源较多的地方进行处理、学习和决策。然而，边缘设备往往受到通信成本的限制，这导致了相当大的功耗，特别是当边缘设备需要通过无线网络发送信息时。提出了一种边缘设备不向中央处理器发送数据，而是向边缘设备发送模型参数（权重）的联合学习方法。每个边缘设备根据接收到的数据修改权重，然后将修改后的权重发送回中央处理器。本提案基于无约束优化方法，利用联邦学习原理，在边缘处理器上开发简单的实时学习算法，并研究电力系统的应用。研究成果将被纳入研究生和本科生的机器学习和信号处理课程。通过夏威夷土著科学和工程指导计划（NHSEMP）和女工程师协会（SWE），将特别关注招募和保留代表性不足的学生。该项目利用联邦学习、自适应信号处理、图信号处理、优化和核方法的原理，在实时分布式学习和决策制定领域取得了进展。这项研究融合了信号处理、数学、统计学和计算机科学等领域。贡献集中在三个方面：算法开发、分析和电网应用。重点是设计简单的在线核算法，既适用于监督学习（回归、预测、分类），也适用于无监督学习（主成分分析、概率密度估计）。算法侧重于边缘计算，使用在线最小二乘核方法的优化方法，使用随机梯度的变体和节点之间的时空关系表示为图。在优化目标函数、收敛性、计算复杂性和通信成本之间进行权衡。然后应用在线分布式核算法检测电网中的不良数据，并为需求响应（DR）程序提供分布式学习能力。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

We are increasingly seeing increasing amounts of dynamic data coming from a variety of different applications ranging from the electric power grid to transportation systems to healthcare networks to social network data. This data comes from a variety of sources including sensor networks and IoT applications. A common feature is that much of this data is that it is distributed (at the edge) and methodologies need to be established to process, learn, and make decisions from this data. One approach is to gather all the data together at a central processor (in the cloud) and to process, learn, and make decisions here where computing resources are greater. The edge devices however are often constrained by communication costs which results in considerable power consumption especially if edge devices need to send information over wireless networks. Federated learning has been proposed where edge devices do not send data to the central processor, but send model parameters (weights) of model to edge devices. Each edge device modifies the weights based on data it receives and then send the modified weights back to the central processor. This proposal develops simple real-time learning algorithms at the edge processor based on unconstrained optimization methods using principles of federated learning and studies applications for power systems. The research results will be incorporated to both graduate and undergraduate courses in machine learning and signal processing. Special attention will be given to recruitment and retention of underrepresented students through the Native Hawaiian Science and Engineering Mentorship Program (NHSEMP) and the Society of Woman Engineers (SWE).The project makes advances in the field of real-time distributed learning and decision making using principles of federated learning, adaptive signal processing, graph signal processing, optimization, and kernel methods. The research is convergent bringing in the fields of signal processing, mathematics, statistics, and computer science together. The contributions are in three areas: algorithm development, analysis, and applicationsto the electric power grid. A focus is to design simple online kernel algorithms that are applicable for both supervised learning (regression, prediction, classification) and unsupervised learning (principal component analysis, probability density estimation). The algorithms focus on edge computing using optimization methods from online least squares kernel methods using variants of stochastic gradient and the temporal and spatial relationships between nodes represented as graphs. Tradeoffs are considered between optimizing objective functions, convergence, computational complexity, and communication costs. The online distributed kernel algorithms are then applied to detect bad data on the power grid and providing distributed learning capabilities for demand response (DR) programs.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.