SHF:Small:Scalable Memory Hierarchies with Fine-Grained QoS Guarantees

多核芯片现在是主流，增加每个芯片的核心数量已经成为提高性能的主要方法。当前的多核依赖于复杂的高速缓存层次结构来缓解主存访问的高延迟、有限带宽和高能量，这些通常限制系统性能。这些片上缓存占用了一半以上的芯片面积，并且大部分缓存空间在所有内核之间共享。共享这种能力具有重大优势，例如提高空间利用率和加速核心到核心通信，但也带来了两个根本问题。首先，内核越多，缓存访问时间越长，消耗的能量也越多，严重限制了可扩展性。其次，并发执行的应用程序会争用此共享缓存容量，这可能导致它们之间不可预测的性能下降。该项目的目标是重新设计该高速缓存层次结构，使其具有高度可伸缩性，并在竞争应用程序之间提供严格的隔离，从而实现端到端的性能保证。如果成功，这项工作将提高未来处理器的性能和能效，使系统拥有比以前更多的内核。此外，这些系统将消除相互竞争的应用程序之间的干扰，并强制执行服务质量保证，即使这些应用程序是延迟关键的。这将大大提高共享计算基础设施的利用率（如云计算服务器），可能节省数十亿美元的IT基础设施和能源消耗。为了有效地实现高可扩展性和服务质量（QoS）保证的双重目标，该项目提出了一种集成的硬件-软件方法，其中硬件公开了一组小而通用的机制来控制缓存分配，并且软件使用这些机制来有效地实现分区和非统一访问策略。在硬件层面，一种新颖的缓存组织提供了数千个细粒度的，空间可配置的分区，实现轻量级的监控和重新配置机制，以有效地指导软件策略，并支持全系统可扩展的缓存一致性便宜。在软件级别，系统级运行时利用此硬件来实现动态数据分类、放置、迁移和复制机制，从而最大限度地提高系统性能和效率，同时对延迟关键型工作负载实施严格的QoS保证，对应用程序透明。结合现有的带宽划分方法，这些技术将通过控制所有片上共享资源（高速缓存、片上网络和存储器控制器）来实施全系统QoS保证。此外，作为该项目的一部分开发的基础设施和基准将被公开发布，允许其他研究人员建立在这项工作的结果，并使课程项目和其他教育活动的发展，在大规模并行计算机体系结构，无论是在麻省理工学院和其他地方。

Multicore chips are now mainstream, and increasing the number of cores per chip has become the primary way to improve performance. Current multicores rely on sophisticated cache hierarchies to mitigate the high latency, limited bandwidth, and high energy of main memory accesses, which often limit system performance. These on-chip caches consume more than half of chip area, and most of this cache space is shared among all cores. Sharing this capacity has major advantages, such as improving space utilization and accelerating core-to-core communication, but poses two fundamental problems. First, with more cores, cache accesses take longer and consume more energy, severely limiting scalability. Second, concurrently executing applications contend for this shared cache capacity, which can cause unpredictable performance degradation among them. The goal of this project is to redesign the cache hierarchy to make it both highly scalable, and to provide strict isolation among competing applications, enabling end-to-end performance guarantees. If successful, this work will improve the performance and energy efficiency of future processors, enabling systems with larger numbers of cores than previously possible. Moreover, these systems will eliminate interference and enforce quality of service guarantees among competing applications, even when those applications are latency-critical. This will enable much higher utilization of shared computing infrastructure (such as cloud computing servers), potentially saving billions of dollars in IT infrastructure and energy consumption.To achieve the dual goals of high scalability and quality-of-service (QoS) guarantees efficiently, this project proposes an integrated hardware-software approach, where hardware exposes a small and general set of mechanisms to control cache allocations, and software uses these mechanisms to implement both partitioning and non-uniform access policies efficiently. At the hardware level, a novel cache organization provides thousands of fine-grained, spatially configurable partitions, implements lightweight monitoring and reconfiguration mechanisms to guide software policies effectively, and supports full-system scalable cache coherence cheaply. At the software level, a system-level runtime leverages this hardware to implement dynamic data classification, placement, migration, and replication mechanisms, maximizing system performance and efficiency, while at the same time enforcing the strict QoS guarantees of latency-critical workloads, transparently to applications. Combined with existing bandwidth partitioning approaches, these techniques will enforce full-system QoS guarantees by controlling all on-chip shared resources (caches, on-chip network, and memory controllers). In addition, the infrastructure and benchmarks developed as part of this project will be publicly released, allowing other researchers to build on the results of this work, and enabling the development of course projects and other educational activities in large-scale parallel computer architecture, both at MIT and elsewhere.