dSyncPS: delayed synchronization for dynamic deployment of distributed machine learning
dSyncPS: delayed synchronization for dynamic deployment of distributed machine learning
复制标题
DOI：
10.1145/3517207.3526981
复制
发表时间：
2022-04
期刊：
影响因子：
--
通讯作者：
Yibo Guo;An Wang
中科院分区：
文献类型：
--
作者：
Yibo Guo;An Wang
在各个领域应用机器学习技术的需求日益增长，推动了复杂机器学习模型的参与。为了满足这一需求，分布式机器学习已经成为模型训练事实上的标准计算范式。机器学习即服务（MLaaS）也作为云服务提供商提供的解决方案出现，以满足这一需求。使用MLaaS，客户可以向服务提供商提交他们的模型和训练数据集，并利用现有的云基础设施进行模型训练和推理。然而，我们发现，对于需要复杂而准确的机器学习模型但只能获得适量数据的最终用户来说，现有的解决方案是不够的。主要问题是缺乏对分布式机器学习任务动态部署的支持。为了解决这个问题，我们提出了一个基于参数服务器的框架，称为dSyncPS，它允许工作节点动态地参与培训。关键思想是，它将参数同步与参数服务器节点中的聚合函数分离，从而导致延迟同步。
The increasing demand of applying machine learning technologies in various domains has driven the involvement of complex machine learning models. To fulfill this demand, distributed machine learning has become the de facto standard computing paradigm for model training. Machine-Learning-as-a-Service (MLaaS) has also emerged as a solution provided by cloud service providers to address this need. With MLaaS, customers can submit their models and training datasets to the service providers, and leverage the existing cloud infrastructure for model training and inference. However, we find that, for end users who require complex and accurate machine learning models but only obtains moderate amount of data, existing solutions are insufficient. The main issue is the lack of support for dynamic deployment of distributed machine learning tasks. To address this issue, we propose a parameter server based framework, called dSyncPS, that allows worker nodes to participate in training dynamically. The key idea is that it separates parameter synchronization from aggregation function in the parameter server nodes, thus resulting in a delayed synchrony.