SHF: Small: Ubiquitous and Transparent Near-data Computing for General Purpose Processors

随着多核处理器的扩展，数据移动和通信的开销成为实现相应性能和能效增益的主要限制。对于依赖大量数据集的“大数据”工作负载尤其如此。一种日益流行的解决方案是在数据存储位置附近执行计算，以避免通信；这是一种称为近数据处理的范式。尽管有潜力，但现有的近数据系统有严重的局限性：它们通常需要程序员的帮助，它们仅限于支持一小部分工作负载，并且它们没有在一致的框架中利用所有近数据处理技术。为了应对这些挑战，该项目开发了新的硬件/软件抽象，可以为通用架构启用近数据处理功能，并且可以由编译器或硬件构建，以避免程序员负担。这项研究的潜在影响是引导微处理器设计以新颖的方式，可以帮助维持预期的指数级性能改进，包括有效地将现有的多核处理器尺寸扩大一个数量级。此外，我们的开源近数据编译器/模拟框架可以促进这个新方向的研究。在教育方面，本项目以基础设施加强课程，让学生在共同设计硬件和软件方面获得跨堆栈经验。最后，该项目制定了一个外展计划，为多所大学的未来研究生提供指导和交流机会。为了实现无处不在和透明的近数据处理目标，本研究的主要技术见解是，一个更强大的近数据抽象将封装程序与每个单独数据结构的交互：其地址模式、相关计算和依赖关系——这种新的程序抽象被称为“光纤”。每个光纤都可以卸载到内存层次结构的某个级别（例如，最后一级缓存或内存），该级别可以对数据位置进行最佳优化，并且所有访问都保持顺序内存语义。光纤之所以具有吸引力，是因为它们的高级特性可用于确定最佳的近数据卸载策略，而且它们的粒度足够粗，可以实现有效的相干性和协调性。为了实现这些目标，该项目解决了五个基本的智力问题：通用性需要什么样的基本卸载策略，特别是考虑到多内存层次结构级别的卸载？如何可能使程序员甚至isa透明的NDP？如何在不被协调消息淹没的情况下启用顺序内存语义？如何在高吞吐量和低开销下执行近数据计算，通过重用通用核心、使用内存处理（PUM）或可重构加速器？最后，考虑到近数据任务的操作数共存的重要性，如何优化NDP的数据放置？总体而言，该项目探索了传统通用处理器以核心为中心范式的根本背离，并将开发新的程序抽象和微架构，以实现基于近数据原理的高效分散计算。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

As multicore processors scale, the overheads of data movement and communication become the primary limitations to achieving commensurate performance and energy-efficiency gains. This is especially true for “big-data” workloads that rely on enormous datasets. One increasingly popular solution is to perform computation near where the data is stored to avoid communication; this is a paradigm called near-data processing. Despite the potential, existing near-data systems have severe limitations: they often require programmer help, they are limited to support a small subset of workloads, and they do not exploit the full range of near-data processing technologies in a coherent framework. To address these challenges, this project develops new hardware/software abstractions that can enable near-data processing capabilities for general purpose architectures, and which can be constructed by a compiler or in hardware to avoid programmer burden. The potential impact of this research is to steer microprocessor design in novel ways that can help sustain expected exponential performance improvements, including efficiently scaling existing multicore processor size by an order-of-magnitude. In addition, our open source near-data compiler/simulation framework can foster research in this new direction. In terms of education, this project enhances courses with the infrastructure to give students cross-stack experience in co-designing hardware and software. Finally, the project develops an outreach program to provide mentorship and networking opportunities to prospective graduate students across multiple universities.Towards the goal of ubiquitous and transparent near-data processing, the primary technical insight of this research is that a more powerful near-data abstraction would encapsulate the program’s interaction with each individual data-structure: its address pattern, associated computation, and dependences -- this new program abstraction is called a “fiber”. Each fiber can be offloaded to a level of the memory hierarchy (e.g. last-level caches or memory) that best optimizes for data locality, and all accesses maintain sequential memory semantics. Fibers are attractive because their high-level properties can be used to determine an optimal near-data offloading strategy, and they are sufficiently coarse grain to enable efficient coherence and coordination. Towards these goals, the project addresses five fundamental intellectual questions: What set of primitive offloading strategies is required for generality, especially considering offloading to multiple memory-hierarchy levels? How is it possible to enable programmer and even ISA-transparent NDP? How to enable sequential memory semantics without being overwhelmed by coordination messages? How to perform near-data computation at high throughput and low overhead, either by reusing general cores, processing-using-memory (PUM), or reconfigurable accelerators? And finally, how to optimize data-placement for NDP, considering the importance of co-locating the operands of near-data tasks? Overall, this project explores a radical departure from the core-centric paradigm of traditional general purpose processors, and will develop new program abstractions and microarchitectures for efficient decentralized computation based on near-data principles.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.