权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CIF: Small: Coding Techniques for Distributed Machine Learning

CIF：小型：分布式机器学习的编码技术

基本信息

批准号：
1910447
负责人：
Jun Li
金额：
$ 50万
依托单位：
Florida International University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-10-01 至 2020-12-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1910447&HistoricalAwards=false
关键词：
CIF Small Coding Techniques Distributed

项目摘要

Modern machine learning models have achieved great success and have been widely deployed across many sectors. As the size of data used to train machine learning models keeps growing, it is now routine to use distributed computing infrastructures such as the cloud. This strategy allows the computation of training to be distributed among a large number of nodes hosted in the cloud, where each node processes a partition of the whole data set. However, the performance of nodes in the cloud is often unreliable, due to system failures, resource contention, load imbalance, etc., and that unreliability can significantly delay the training process. This project pursues a coding-based framework that not only tolerates the effects of faulty nodes, but also further enhances the performance of machine learning training by dynamically taking advantage of the resources available on all nodes, whether they are faulty or not. The outcomes of this project should lead to a significant performance boost for distributed training of machine learning models.To enable the efficient use of distributed computing across unreliable infrastructure for training machine learning models from big data sets, the technical objectives of this project are divided into three levels. This project will first study coding theory for distributed matrix multiplication, a universal operation in various machine learning algorithms, and propose a coding framework with both fault tolerance and a significant performance boost. This framework will then be applied into parameter servers at the architecture level and deep neural networks at the model level, respectively. Combining these three parts, this work will lead to a practical coding framework that can efficiently scale out computation on heterogeneous unreliable nodes, where the coding schemes will be applied to distributed machine learning at different levels including fundamental arithmetic, architectures, and models.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

现代机器学习模型已经取得了巨大的成功，并在许多领域得到了广泛的应用。随着用于训练机器学习模型的数据规模不断增长，现在使用云计算等分布式计算基础设施已成为常规。这种策略允许训练的计算分布在云中托管的大量节点中，其中每个节点处理整个数据集的一个分区。然而，云中节点的性能往往是不可靠的，由于系统故障、资源争用、负载不平衡等原因，并且这种不可靠性会显著地延迟训练过程。该项目追求一个基于编码的框架，不仅容忍故障节点的影响，而且通过动态利用所有节点上可用的资源（无论它们是否有故障）来进一步提高机器学习训练的性能。该项目的成果将显著提升机器学习模型分布式训练的性能。为了在不可靠的基础设施上有效使用分布式计算，从大数据集中训练机器学习模型，该项目的技术目标分为三个级别。该项目将首先研究分布式矩阵乘法的编码理论，这是各种机器学习算法中的通用操作，并提出一个具有容错性和显着性能提升的编码框架。然后，该框架将分别应用于架构级别的参数服务器和模型级别的深度神经网络。结合这三个部分，这项工作将导致一个实用的编码框架，可以有效地扩展异构不可靠节点上的计算，其中编码方案将应用于不同层次的分布式机器学习，包括基础算法，架构，该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的评估支持影响审查标准。