Distributed Networked Real-Time Learning
Distributed Networked Real-Time Learning
复制标题
DOI：
10.1109/tcns.2020.3029992
复制
发表时间：
2020-09
影响因子：
4.2
通讯作者：
Alfredo García;Luochao Wang;Jeff Huang;Lingzhou Hong
中科院分区：
文献类型：
--
作者：
Alfredo García;Luochao Wang;Jeff Huang;Lingzhou Hong
许多机器学习算法都是在假设数据集已经以批量形式可用的情况下开发的。然而，在许多应用领域中，数据只能通过不同地理位置的计算节点按顺序随时间推移提供。在这篇文章中，我们考虑了当流数据不能及时传输到单个位置时学习模型的问题。在这种情况下，需要一种分布式的学习架构，这种架构依赖于互连的“本地”节点网络。我们提出了一个分布式的计划，其中每个本地节点实现随机梯度更新的基础上，本地数据流。为了确保鲁棒估计，使用网络正则化惩罚来维持模型集合中的内聚性度量。我们表明，合奏平均近似一个固定点，并表征在何种程度上，个别模型不同于合奏平均。我们将结果与联邦学习进行比较，得出结论，该方法对数据流（数据速率和估计质量）的异质性更具鲁棒性。我们通过基于卷积神经网络的深度学习模型在图像分类中的应用来说明结果。
Many machine learning algorithms have been developed under the assumption that datasets are already available in batch form. Yet, in many application domains, data are only available sequentially overtime via compute nodes in different geographic locations. In this article, we consider the problem of learning a model when streaming data cannot be transferred to a single location in a timely fashion. In such cases, a distributed architecture for learning which relies on a network of interconnected “local” nodes is required. We propose a distributed scheme in which every local node implements stochastic gradient updates based upon a local data stream. To ensure robust estimation, a network regularization penalty is used to maintain a measure of cohesion in the ensemble of models. We show that the ensemble average approximates a stationary point and characterizes the degree to which individual models differ from the ensemble average. We compare the results with federated learning to conclude that the proposed approach is more robust to heterogeneity in data streams (data rates and estimation quality). We illustrate the results with an application to image classification with a deep learning model based upon convolutional neural networks.