CAREER: Architecting a Hardware-Software Co-Designed Data Management System for Heterogeneous Memory Computers

数据处理是社会不可或缺的一部分，我们的计算系统需要处理的数据量正在迅速增长。例如，当今的一些人工智能（AI）和机器学习（ML）工作负载具有数十亿个参数，需要TB的工作内存进行训练和推理。为了有效地处理这些海量数据，必须首先将数据从远程随机存取存储器移动到离处理器较近的存储器。在当今的系统中，这种数据移动本身是性能和能量/功率方面最昂贵的计算方面之一。为了提高计算系统的效率，这个CAREER项目旨在回答“数据应该放在哪里，什么时候应该从一种内存类型移动到另一种内存类型？”推动这一CAREER提案的主要观点是，弥合知道如何使用数据的应用程序与直接与数据交互的硬件之间的语义差距，将实现一组新的优化。利用这一洞察力，这个CAREER项目将开发一个新的数据移动接口，将数据移动机制与数据移动策略分开。这种关注点的分离将使程序员提示和运行时导向的数据移动成为可能，这将显著提高异构内存系统的性能和效率。这个CAREER项目将提高包括AI/ML在内的各种应用的效率和性能，这些应用越来越依赖于高内存性能和大内存容量。为了扩大影响，在这个项目中开发的技术将在软件中实现，作为硬件-软件协同设计，并作为硬化逻辑。此外，软件模型，系统和硬件原型将在开发时开源发布。这个职业生涯项目也将为加州大学戴维斯分校的可持续本科研究实习计划奠定基础。该计划将提供开源软件开发经验，职业发展机会和实际操作的硬件-软件协同设计经验。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Data processing is integral to society, and the amount of data our computing systems are asked to process is growing rapidly. For example, some of today’s artificial intelligence (AI) and machine learning (ML) workloads have billions of parameters requiring terabytes of working memory for training and inference. To efficiently process this deluge of data, the data must first be moved from distant random-access memory to memory closer to the processor. In today’s systems, this data movement itself is one of the most expensive aspects of computation in terms of both performance and energy/power. To improve the efficiency of computing systems, this CAREER project aims to answer the question “where should data be placed and when should it move from one memory type to another?” The main insight driving this CAREER proposal is that bridging the semantic gap between the application which knows how the data is used and the hardware which directly interacts with the data will enable a new set of optimizations. Leveraging this insight, this CAREER project will develop a new interface for data movement which separates the data movement mechanism from the data movement policies. This separation of concerns will enable programmer hints and runtime-directed data movement which will significantly increase the performance and efficiency of heterogeneous memory systems.This CAREER project will improve the efficiency and performance of a wide variety of applications including AI/ML which increasingly depend on both high memory performance and large memory capacities. To broaden the impact, the techniques developed in this project will be realized in software, as a hardware-software codesign, and as hardened logic. Further, the software models, systems, and hardware prototypes will be released open source as they are developed. This CAREER project will also lay the foundation for a sustainable undergraduate research internship program at UC Davis. This program will give experience with open-source software development, career development opportunities, and hands-on hardware-software co-design experience.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.