XPS: FULL: Broad-Purpose, Aggressively Asynchronous and Theoretically Sound Parallel Large-scale Machine Learning

XPS：FULL：用途广泛、积极异步且理论上合理的并行大规模机器学习

• 批准号: 1629559
• 负责人: Eric Xing
• 金额: $ 62.54万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-01 至 2020-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1629559&HistoricalAwards=false
• 关键词: XPS; FULL; Broad; Purpose; Aggressively

Many artificial intelligence (AI) applications such as image understanding and natural language processing rely on Machine Learning (ML) methods to automatically extract valuable knowledge from Big Data (Big Learning). Efficient ML requires not only expertise in advanced mathematical models and algorithms, but also experiences with large computer clusters where issues such as machine failures, memory/network bottlenecks, inter-machine latencies must be properly handled through complex system programming. Such demand on "dual skill" often prevents democratizing large-scale AI to wide user communities, and necessitates a new framework that bridges ML and the distributed computing environment of a cluster with a single-machine-like simple interface, allowing ML practitioners to be agnostic about the backend details, and able to quickly prototype or deploy ML programs on clusters. Solutions to such a need remain rare. In this project the PIs develop a new general purpose framework for ML on distributed systems, offering highly efficient and theoretically justified protocols (e.g. communication, scheduling, and partitioning functions) to orchestrate a heterogeneous computer cluster to become programmable and act like a single big computer, and execute distributed ML programs correctly and at a speed orders of magnitude faster than current systems such as Hadoop and Spark. With this new framework, data scientists will be able to conduct ML analytics with complex models on massive data without the need for dedicated engineering and infrastructure teams, allowing Big Learning more readily accessible to society. Specifically, over a four year span, the proposed research focuses on three technical aims: (1) Building a System Framework for Big Learning, by developing a new architecture that supports both data- and model-parallel execution of large ML programs, using intelligent scheduler, parameter server, and consistency controller that are configurable to provide flexible options for model/data parallelization, synchronization schemes, load balance, fault tolerance, and multi-instance tenancy; (2) Building a Multi-Level-Abstraction Programming Interface, which supports easy parallel programming of both basic and advanced ML algorithms for large-scale applications; and (3)Conducting theoretical analysis of distributed ML algorithms on the proposed system, based on unique insights such as block consistency and error-tolerance under bounded synchronism. The goal is to develop a system framework to achieve general, automatic, and effective parallelization of ML programs.

许多人工智能(AI)应用，如图像理解和自然语言处理，都依赖于机器学习(ML)方法来自动从大数据(Big Learning)中提取有价值的知识。高效的ML不仅需要高级数学模型和算法方面的专业知识，还需要处理大型计算机集群的经验，在这些集群中，机器故障、内存/网络瓶颈、机器间延迟等问题必须通过复杂的系统编程得到适当处理。这种“双重技能”的需求往往阻碍了大规模人工智能向广泛的用户社区的民主化，并需要一个新的框架来连接ML和集群的分布式计算环境，使用单机般的简单界面，允许ML实践者对后端细节不可知，并能够在集群上快速制作或部署ML程序的原型。针对这种需求的解决方案仍然很少。在这个项目中，PI为分布式系统上的ML开发了一个新的通用框架，提供了高效和理论上合理的协议(例如，通信、调度和分区功能)来协调不同的计算机集群，使其变得可编程并像一台大型计算机一样运行，并正确地执行分布式ML程序，并且速度比当前的系统(如Hadoop和Spark)快几个数量级。有了这个新的框架，数据科学家将能够使用复杂的模型对海量数据进行ML分析，而不需要专门的工程和基础设施团队，使Big Learning更容易为社会所接受。具体地说，在四年的时间里，建议的研究集中在三个技术目标上：(1)通过开发一个支持大型ML程序的数据并行和模型并行执行的新体系结构，使用可配置的智能调度器、参数服务器和一致性控制器，为模型/数据并行化、同步方案、负载平衡、容错和多实例租用提供灵活的选项，从而构建一个用于大学习的系统框架；(2)构建一个多层次抽象编程接口，它支持大规模应用程序的基本和高级ML算法的简单并行编程；(3)基于有界同步下的块一致性和容错性等独特见解，对分布式最大似然算法进行了理论分析。目标是开发一个系统框架，以实现ML程序的通用、自动和有效的并行化。