SHF: Small: Software/Hardware Acceleration Architectures for Low-Tail-Latency QoS Provisioning Based Data Centers

数据科学时代正在到来，来自社交媒体、环境监测、电子健康、国防、科学/工程进步等的数据爆炸式增长，推动了信息技术领域的快速发展。作为大数据的基础支柱，数据中心在有效收集、存储、检索、分类和处理大型数据集方面发挥着至关重要的作用。此外，数据中心中的大量数据以及现代计算机技术的快速发展，推动了人工智能（AI）机器学习（ML）的持续繁荣。虽然ML的目标是从数据中自动学习有用的属性，以进行准确和及时的随机决策，但越来越需要以实时方式进行这种决策。因此，基于AI的交互式数据中心中最重要的服务之一是如何有效地处理计算密集型和时间敏感型多媒体（例如，视频、音频）数据，并提供基于AI的决策服务。然而，由于有限的计算和存储能力，软件/硬件资源的可用性的随机不确定性，以及数据中心中的统计复用切换，基于人工智能的交互式数据中心的大容量实时服务的确定性延迟限制的要求往往是不可行的。因此，PI建议扩展和应用统计延迟有界服务质量（QoS）提供理论作为支持实时决策服务的替代解决方案，其目标是以较小的违规概率保证有界延迟，从而显着减少当前基于AI的交互式数据中心中的处理延迟。这些要求开发各种软件/硬件加速器，以保证不同的延迟限制的QoS要求。本研究的目的是系统地调查基本的和具有挑战性的问题，如何扩展，应用和实施的统计延迟限制的QoS提供理论，在支持实时，交互式和决策服务基于人工智能的交互式数据中心。 虽然统计延迟有界QoS供应理论已经被证明是用于支持在移动的计算网络上的时间敏感的多媒体传输的强大技术和有用的性能度量，但是如何有效地扩展和实现该技术/性能度量以用于统计上界尾延迟，尾延迟是指示延迟有界QoS性能的最坏情况的延迟，在基于人工智能的交互式数据中心服务中施加的这些限制既没有得到很好的理解，也没有得到彻底的研究。为了克服上述挑战，采用各种新兴的计算机软件/硬件技术，本项目提出开发一套基于人工智能的混合软件/硬件加速架构，算法和方案，以支持基于多核人工智能的交互式数据中心服务的低尾延迟QoS提供，同时减少并行和分布式数据中心带来的计算工作量和复杂性。所提出的框架主要基于开发用于软件和硬件设计和优化的新型加速架构，以通过最小化跨处理器和存储器的指令和数据移动和处理来显著提高计算效率。利用统计延迟约束QoS提供理论和基于AI的计算加速器的独特新颖功能和技术，一些QoS使能引擎构成了该项目的主要基础。更具体地说，研究主要集中在以下紧密耦合的研究任务。(1)开发基于深度学习的内存处理（PIM）系统（PIM QoS引擎），以加快应用程序分类的培训。(2)开发基于深度学习的应用程序编码/聚合机制，然后将编码向量与训练的分析输出进行比较，以分类/聚合应用程序。(3)开发分层缓存分区架构，通过基于负载配置文件对应用程序进行群集，在统计上限制数据中心服务的尾部延迟。(4)开发精确的尾延迟QoS性能预测模型/指标和监控系统，以保证高优先级并发运行应用的低尾延迟的统计延迟限制QoS。(5)开发建模和分析技术，以及仿真工具/测试平台，以验证和评估拟议架构，框架，协议/算法和方案的性能。项目的研究旨在造福国民经济、环境和社会。此外，该项目是很好地集成与PI的新的研究生和本科生数据中心相关的课程/课程在得克萨斯州A M大学的发展。该项目的重要发现将通过期刊、会议和网站等渠道传播给研究界。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The era of data science is underway, with an explosion of data from social media, environmental monitoring, E-health, national defense, sciences/engineering advances, etc., driving a very fast-growing information-technology sector. As a foundational pillar for big data, data centers play a crucially important role in efficiently collecting, storing, retrieving, classifying, and processing large datasets. In addition, these tremendous volumes of data in data centers, as well as the rapid advances of modern computer techniques, have propelled the ongoing boom of machine learning (ML) from artificial intelligence (AI). While ML aims at automatically learning useful properties from data for accurate and timely stochastic decision making, there is an increasing need for this decision making to occur in a real-time fashion. Thus, one of the most important services in an AI-based interactive data center is how to efficiently process computation-intensive and time-sensitive multimedia (e.g., video, audio) data and provide AI-based decision-making services. However, because of limited computing and storage capabilities, random uncertainties of availability for software/hardware resources, and statistical multiplex switching in data centers, the deterministic delay-bounded requirements for high-volume real-time services of AI-based interactive data-centers are often infeasible. Thus, the PI proposes to extend and apply the statistical delay-bounded quality-of-service (QoS) provisioning theory as an alternative solution to support real-time decision-making services, where the goal is to guarantee bounded delay with a small violation probability, therefore significantly reducing the processing delays currently found in AI-based interactive data-centers. These demand various software/hardware accelerators to be developed to guarantee diverse delay-bounded QoS requirements. The objective of this research is to systematically investigate fundamental and challenging issues on how to extend, apply, and implement the statistical delay-bounded QoS provisioning theory in supporting real-time, interactive, and decision-making services over AI-based interactive data centers. While the statistical delay-bounded QoS provisioning theory has been shown to be a powerful technique and useful performance metric for supporting time-sensitive multimedia transmissions over mobile computing networks, how to efficiently extend and implement this technique/performance-metric for statistically upper-bounding the tail-Latency, which is the worst-case latency dictating delay-bounded QoS performances, imposed in the AI-based interactive data center services has neither been well understood nor thoroughly studied. To overcome the above challenges, employing various emerging computer software/hardware technologies, this project proposes to develop a set of AI-based hybrid software/hardware acceleration architectures, algorithms, and schemes to support the low-tail-latency QoS provisioning for multi-core AI-based interactive data-center services, while reducing the computational workloads and complexities introduced by parallel and distributed data centers. The proposed framework is mainly based on developing novel acceleration architectures for both software and hardware designs and optimizations to significantly boost computing efficiencies through minimizing instruction and data movement and processing across processors and memories. Leveraging the unique novel features and techniques of the statistical delay-bounded QoS provisioning theory and AI-based computing accelerators, a number of QoS-enabling engines constitute the main foundation of this project. More specifically, the research focuses mainly on the following closely coupled research tasks. (1) Develop deep-learning-based processing-in-memory (PIM) systems (PIM QoS-enabling engine) to accelerate training for applications classifications. (2) Develop deep-learning-based application-encoding/aggregating mechanisms and then compare the encoded vectors with trained profiling outputs to classify/aggregate applications. (3) Develop hierarchical cache-partitioning architectures to statistically upper-bound the tail-latency of data-center services by clustering applications based on their load profiles. (4) Develop the precise tail-latency QoS performance-prediction models/metrics and monitoring systems to guarantee the statistical delay-bounded QoS for low tail latency of the higher-priority co-running applications. (5) Develop modeling and analytical techniques, and simulation tools/testbeds, to validate and evaluate the performance for the proposed architectures, frameworks, protocols/algorithms, and schemes. The projects' research intends to benefit the national economy, environment, and society. Also, this project is well integrated with PI’s developments of new graduate and undergrad data-center-relevant curricula/courses at Texas A&M University. The important findings of this project are to be disseminated to the research community through the avenues of journals, conferences, and websites.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.