BIGDATA: Collaborative Research: F: Nomadic Algorithms for Machine Learning in the Cloud

BIGDATA：协作研究：F：云中机器学习的游牧算法

• 批准号: 1546459
• 负责人: Manfred Warmuth
• 金额: $ 59.63万
• 依托单位: University of California-Santa Cruz
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1546459&HistoricalAwards=false
• 关键词: BIGDATA; Collaborative; Research; Nomadic; Algorithms

With an ever increasing ability to collect and archive data, massive data sets are becoming increasingly common. These data sets are often too big to fit into the main memory of a single computer, and so there is a great need for developing scalable and sophisticated machine learning methods for their analysis. In particular, one has to devise strategies to distribute the computation across multiple machines. However, stochastic optimization and inference algorithms that are so effective for large-scale machine learning appear to be inherently sequential.The main research goal of this project is to develop a novel "nomadic" framework that overcomes this barrier. This will be done by showing that many modern machine learning problems have a certain "double separability" property. The aim is to exploit this property to develop convergent, asynchronous, distributed, and fault tolerant algorithms that are well-suited for achieving high performance on commodity hardware that is prevalent on today's cloud computing platforms. In particular, over a four year period, the following will be developed: (i) parallel stochastic optimization algorithms for the multi-machine cloud computing setting, (ii) theoretical guarantees of convergence, (iii) open source code under a permissive license, (iv) application of these techniques to a variety of problem domains such as topic models and mixture models. In addition, a cohort of students who can transfer their skills to both industry and academia will be trained, and a graduate level course on scalable machine learning will be developed. The proposed research will enable practitioners in different application areas to quickly solve their big data problems. The results of the project will be disseminated widely through papers and open source software. Course material will be developed for the education of students in the area of Scalable Machine Learning, and the course will be co-taught at UCSC and UT Austin. The project will recruit women and minority students.

随着收集和归档数据的能力不断提高，海量数据集变得越来越普遍。这些数据集通常太大，无法放入单个计算机的主内存中，因此非常需要开发可扩展和复杂的机器学习方法进行分析。特别是，人们必须设计策略来将计算分布在多台机器上。然而，对于大规模机器学习如此有效的随机优化和推理算法似乎是固有的顺序。该项目的主要研究目标是开发一种新颖的“游牧”框架，以克服这一障碍。 这将通过展示许多现代机器学习问题具有某种“双重可分性”属性来实现。其目的是利用这一属性来开发收敛，异步，分布式和容错算法，这些算法非常适合在当今云计算平台上流行的商品硬件上实现高性能。特别是，在四年的时间里，将开发以下内容：（i）多机云计算设置的并行随机优化算法，（ii）收敛的理论保证，（iii）许可证下的开源代码，（iv）将这些技术应用于各种问题领域，如主题模型和混合模型。此外，还将培训一批能够将技能转移到工业界和学术界的学生，并将开发一门关于可扩展机器学习的研究生课程。这项研究将使不同应用领域的从业者能够快速解决他们的大数据问题。 该项目的成果将通过文件和开放源码软件广泛传播。课程材料将开发用于可扩展机器学习领域的学生教育，该课程将在UCSC和UT Austin共同授课。该项目将招收妇女和少数民族学生。