CAREER: Architecting a Hardware-Software Co-Designed Data Management System for Heterogeneous Memory Computers

数据处理是社会不可或缺的一部分，我们的计算系统需要处理的数据量正在迅速增长。例如，今天的一些人工智能（AI）和机器学习（ML）工作负载有数十亿个参数，需要tb级的工作记忆来进行训练和推理。为了有效地处理大量数据，必须首先将数据从远程随机存取存储器移到离处理器较近的存储器中。在今天的系统中，这种数据移动本身是计算中最昂贵的方面之一，无论是在性能还是能源/功率方面。为了提高计算系统的效率，这个CAREER项目旨在回答“数据应该放在哪里，什么时候应该从一种存储类型转移到另一种存储类型？”驱动这个CAREER建议的主要观点是，在了解如何使用数据的应用程序和直接与数据交互的硬件之间架起一座语义鸿沟的桥梁，这将使一组新的优化成为可能。利用这种洞察力，这个CAREER项目将为数据移动开发一个新的接口，该接口将数据移动机制与数据移动策略分开。这种关注点分离将支持程序员提示和面向运行时的数据移动，这将显著提高异构内存系统的性能和效率。这个CAREER项目将提高包括AI/ML在内的各种应用的效率和性能，这些应用越来越依赖于高内存性能和大内存容量。为了扩大影响，本项目中开发的技术将在软件中实现，作为硬件软件协同设计，并作为硬化逻辑。此外，软件模型、系统和硬件原型将在开发过程中以开源的方式发布。这个CAREER项目也将为加州大学戴维斯分校可持续的本科生研究实习项目奠定基础。该计划将提供开源软件开发经验，职业发展机会，以及动手硬件软件协同设计经验。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Data processing is integral to society, and the amount of data our computing systems are asked to process is growing rapidly. For example, some of today’s artificial intelligence (AI) and machine learning (ML) workloads have billions of parameters requiring terabytes of working memory for training and inference. To efficiently process this deluge of data, the data must first be moved from distant random-access memory to memory closer to the processor. In today’s systems, this data movement itself is one of the most expensive aspects of computation in terms of both performance and energy/power. To improve the efficiency of computing systems, this CAREER project aims to answer the question “where should data be placed and when should it move from one memory type to another?” The main insight driving this CAREER proposal is that bridging the semantic gap between the application which knows how the data is used and the hardware which directly interacts with the data will enable a new set of optimizations. Leveraging this insight, this CAREER project will develop a new interface for data movement which separates the data movement mechanism from the data movement policies. This separation of concerns will enable programmer hints and runtime-directed data movement which will significantly increase the performance and efficiency of heterogeneous memory systems.This CAREER project will improve the efficiency and performance of a wide variety of applications including AI/ML which increasingly depend on both high memory performance and large memory capacities. To broaden the impact, the techniques developed in this project will be realized in software, as a hardware-software codesign, and as hardened logic. Further, the software models, systems, and hardware prototypes will be released open source as they are developed. This CAREER project will also lay the foundation for a sustainable undergraduate research internship program at UC Davis. This program will give experience with open-source software development, career development opportunities, and hands-on hardware-software co-design experience.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.