EAGER: Scaling Up Machine Learning with Virtual Memory

TB或PB级的大型数据集越来越常见，这就需要新型的可扩展机器学习方法。虽然最先进的技术通常使用复杂的设计，专门的方法来存储和处理大型数据集，但该项目提出了一种最低限度的方法，通过利用所有现代操作系统上的基本虚拟内存功能，将大型数据集加载到虚拟内存空间中，否则这些数据集太大，无法容纳在计算机的主内存中。这一主要思想将使开发人员能够轻松地处理大型数据集，就像它们是内存中的数据一样，使他们能够创建更容易开发和维护的机器学习软件，而且速度更快，可扩展性更强。开发人员将实现更高的工作效率，并减少编程错误;公司将降低运营成本;研究人员将创新方法，而不会陷入实现细节和可扩展性问题。提出的想法可能会对科学，教育和技术领域的工业和学术界产生深远的影响，因为他们在大型数据集上应用机器学习时面临越来越多的挑战。提出的想法还将有助于培养下一代科学家和工程师，让学生学习以更简单的方式处理大型数据集。由于虚拟内存在现代设备和操作系统上普遍可用，因此所提出的想法也将适用于移动的低功耗设备，使它们能够以前所未有的规模和速度执行计算。该项目研究了一种基本的，激进的方法来扩展基于虚拟内存的机器学习算法，这种方法可能更容易编码和维护，但是目前在单机和多机分布式方法中利用不足。这项研究旨在深入了解这个激进的想法，它的好处和局限性，以及这些结果在多大程度上适用于各种设置，关于数据集，内存大小，页面大小（例如，从默认的4KB到实现兆兆字节虚拟存储器空间的巨型2 MB页面），以及体系结构（例如，在分布式共享内存文件系统（如Lustre，支持大型计算机集群上的分页和虚拟内存）上进行测试。研究人员将建立在他们对图形算法的初步研究基础上，这些算法已经证明了比最先进的方法有显著的速度提高;他们将把他们的方法扩展到广泛的机器学习和数据挖掘算法。他们还将开发数学模型和系统方法，以基于跨平台，数据集和语言的广泛评估来分析和预测算法性能和能源使用。欲了解更多信息，请访问项目网站：http://poloclub.gatech.edu/mmap/。

Large datasets in terabytes or petabytes are increasingly common, calling for new kinds of scalable machine learning approaches. While state-of-the-art techniques often use complex designs, specialized methods to store and work with large datasets, this project proposes a minimalist approach that forgoes such complexities, by leveraging the fundamental virtual memory capability found on all modern operating systems, to load into the virtual memory space the large datasets that are otherwise too large to fit in the computer's main memory. This main idea will allow developers to easily work with large datasets as if they were in-memory data, enabling them to create machine learning software that is significantly easier to develop and maintain, yet faster and more scalable. Developers will achieve higher work efficiency and make fewer programming errors; companies will reduce operating costs; and researchers will innovate methodology without getting bogged down by implementation details and scalability concerns. The proposed ideas could make a far-reaching impact on industry and academia, in science, education, and technology, as they face increasing challenges in applying machine learning on large datasets. The proposed ideas will also help train the next generation of scientists and engineers by allowing students to learn to work with large datasets in significantly simpler ways. As virtual memory is universally available on modern devices and operating systems, the proposed ideas will also work on mobile, low-power devices, enabling them to perform computation at unprecedented scales and speed.This project investigates a fundamental, radical way to scale up machine learning algorithms based on virtual memory, one that may be easier to code and maintain, but currently under-utilized in by both single-machine and multi-machine distributed approaches. This research aims to develop deep understanding of this radical idea, its benefits and limitations, and to what extent these results apply in various settings, with respect to datasets, memory sizes, page sizes (e.g., from the default 4KB to the jumbo 2MB pages that enable terabyes of virtual memory space), and architectures (e.g., testing on distributed shared memory file systems like Lustre that support paging and virtual memory over large computer clusters). The researchers will build on their preliminary work on graph algorithms that already demonstrates significant speed-up over state-of-the-art approaches; they will extend their approach to a wide range of machine learning and data mining algorithms. They will also develop mathematical models and systematic approaches to profile and predict algorithm performance and energy usage based on extensive evaluation across platforms, datasets, and languages. For further information, see the project web site at: http://poloclub.gatech.edu/mmap/.