EAGER: Scaling Up Machine Learning with Virtual Memory

以TB或PB为单位的大数据集越来越普遍，这就需要新的可扩展的机器学习方法。虽然最先进的技术经常使用复杂的设计和专门的方法来存储和处理大型数据集，但该项目提出了一种最低限度的方法，通过利用所有现代操作系统上的基本虚拟内存功能，将原本太大而无法放入计算机主内存的大型数据集加载到虚拟内存空间中，从而放弃了这种复杂性。这一主要思想将允许开发人员轻松地处理大型数据集，就像它们是内存中的数据一样，使他们能够创建更容易开发和维护的机器学习软件，但速度更快，可伸缩性更强。开发人员将获得更高的工作效率，并减少编程错误；公司将降低运营成本；研究人员将创新方法，而不会陷入实现细节和可扩展性问题的泥潭。提出的想法可能会对工业界和学术界以及科学、教育和技术产生深远影响，因为他们在大数据集上应用机器学习方面面临着越来越多的挑战。提出的想法还将帮助培训下一代科学家和工程师，让学生学习以明显更简单的方式处理大型数据集。由于虚拟内存在现代设备和操作系统上普遍可用，提出的想法也将适用于移动、低功耗设备，使它们能够以前所未有的规模和速度执行计算。该项目研究了一种基本的、根本的方法来扩展基于虚拟内存的机器学习算法，这种算法可能更易于编码和维护，但目前在单机和多机分布式方法中都没有得到充分利用。这项研究旨在深入理解这一激进的想法、它的好处和局限性，以及这些结果在多大程度上适用于各种环境，涉及数据集、内存大小、页面大小(例如，从默认的4KB到支持TB级虚拟内存空间的巨型2MB页面)和体系结构(例如，在支持大型计算机集群上的分页和虚拟内存的Lustre等分布式共享内存文件系统上进行测试)。研究人员将在他们关于图形算法的初步工作的基础上，已经证明了比最先进的方法有显著的速度；他们将把他们的方法扩展到广泛的机器学习和数据挖掘算法。他们还将开发数学模型和系统方法，以基于跨平台、数据集和语言的广泛评估来分析和预测算法性能和能源使用情况。有关更多信息，请参阅项目网站：http://poloclub.gatech.edu/mmap/.

Large datasets in terabytes or petabytes are increasingly common, calling for new kinds of scalable machine learning approaches. While state-of-the-art techniques often use complex designs, specialized methods to store and work with large datasets, this project proposes a minimalist approach that forgoes such complexities, by leveraging the fundamental virtual memory capability found on all modern operating systems, to load into the virtual memory space the large datasets that are otherwise too large to fit in the computer's main memory. This main idea will allow developers to easily work with large datasets as if they were in-memory data, enabling them to create machine learning software that is significantly easier to develop and maintain, yet faster and more scalable. Developers will achieve higher work efficiency and make fewer programming errors; companies will reduce operating costs; and researchers will innovate methodology without getting bogged down by implementation details and scalability concerns. The proposed ideas could make a far-reaching impact on industry and academia, in science, education, and technology, as they face increasing challenges in applying machine learning on large datasets. The proposed ideas will also help train the next generation of scientists and engineers by allowing students to learn to work with large datasets in significantly simpler ways. As virtual memory is universally available on modern devices and operating systems, the proposed ideas will also work on mobile, low-power devices, enabling them to perform computation at unprecedented scales and speed.This project investigates a fundamental, radical way to scale up machine learning algorithms based on virtual memory, one that may be easier to code and maintain, but currently under-utilized in by both single-machine and multi-machine distributed approaches. This research aims to develop deep understanding of this radical idea, its benefits and limitations, and to what extent these results apply in various settings, with respect to datasets, memory sizes, page sizes (e.g., from the default 4KB to the jumbo 2MB pages that enable terabyes of virtual memory space), and architectures (e.g., testing on distributed shared memory file systems like Lustre that support paging and virtual memory over large computer clusters). The researchers will build on their preliminary work on graph algorithms that already demonstrates significant speed-up over state-of-the-art approaches; they will extend their approach to a wide range of machine learning and data mining algorithms. They will also develop mathematical models and systematic approaches to profile and predict algorithm performance and energy usage based on extensive evaluation across platforms, datasets, and languages. For further information, see the project web site at: http://poloclub.gatech.edu/mmap/.