BIGDATA: Collaborative Research: F: Nomadic Algorithms for Machine Learning in the Cloud

随着收集和归档数据的能力不断增强，海量数据集正变得越来越常见。这些数据集往往太大，无法放入单台计算机的主存储器中，因此非常需要开发可扩展和复杂的机器学习方法来分析它们。特别是，人们必须设计策略来将计算分布在多台机器上。然而，对于大规模机器学习如此有效的随机优化和推理算法似乎具有内在的顺序性，本项目的主要研究目标是开发一种克服这一障碍的新颖的“游牧”框架。这将通过展示许多现代机器学习问题具有一定的“双重可分性”性质来实现。其目的是利用这一特性来开发收敛、异步、分布式和容错算法，这些算法非常适合在当今云计算平台上流行的商用硬件上实现高性能。特别是，在四年的时间里，将开发以下内容：(I)用于多机器云计算环境的并行随机优化算法，(Ii)收敛的理论保证，(Iii)在许可许可下的开放源代码，(Iv)将这些技术应用于各种问题领域，如主题模型和混合模型。此外，还将培训一批能够将他们的技能转移到工业和学术界的学生，并将开发一门关于可扩展机器学习的研究生水平课程。拟议的研究将使不同应用领域的从业者能够快速解决他们的大数据问题。该项目的成果将通过论文和开放源码软件广泛传播。课程材料将开发用于可扩展机器学习领域的学生教育，该课程将在加州大学洛杉矶分校和加州大学奥斯汀分校共同教授。该项目将招收女性和少数族裔学生。

With an ever increasing ability to collect and archive data, massive data sets are becoming increasingly common. These data sets are often too big to fit into the main memory of a single computer, and so there is a great need for developing scalable and sophisticated machine learning methods for their analysis. In particular, one has to devise strategies to distribute the computation across multiple machines. However, stochastic optimization and inference algorithms that are so effective for large-scale machine learning appear to be inherently sequential.The main research goal of this project is to develop a novel "nomadic" framework that overcomes this barrier. This will be done by showing that many modern machine learning problems have a certain "double separability" property. The aim is to exploit this property to develop convergent, asynchronous, distributed, and fault tolerant algorithms that are well-suited for achieving high performance on commodity hardware that is prevalent on today's cloud computing platforms. In particular, over a four year period, the following will be developed: (i) parallel stochastic optimization algorithms for the multi-machine cloud computing setting, (ii) theoretical guarantees of convergence, (iii) open source code under a permissive license, (iv) application of these techniques to a variety of problem domains such as topic models and mixture models. In addition, a cohort of students who can transfer their skills to both industry and academia will be trained, and a graduate level course on scalable machine learning will be developed. The proposed research will enable practitioners in different application areas to quickly solve their big data problems. The results of the project will be disseminated widely through papers and open source software. Course material will be developed for the education of students in the area of Scalable Machine Learning, and the course will be co-taught at UCSC and UT Austin. The project will recruit women and minority students.