SHF: Medium: WHOLEPRO: An Online, Holistic Job Scheduling and Resource Provisioning Framework for Datacenter Architectures and Applications

数据中心服务的最终设计目标是实现以用户为中心的服务，满足个人用户的各种体验质量（QoE）需求，同时允许在用户之间进行优化和公平的计算和网络资源共享。然而，实现这一目标绝非易事。现有的数据中心作业调度和资源供应机制是自底向上的点解决方案，只处理相关设计挑战的部分方面，因此无法实现上述目标。所提议的研究将开发一种涉及整个数据中心系统堆栈的自上而下的整体方法，旨在实现双重目标。它从最顶层开始，即用户应用程序级别的QoE或用户实用程序需求。然后向下工作，将用户级需求映射到数据中心系统级资源需求，再加上数据中心计算和网络资源约束，为建议的用户效用感知优化框架设置边界条件。最后，将为最先进的分布式数据中心开发一个基于软件的示例设计和实现所建议的解决方案。成功实现这一目标将为具有不同QoE需求的广泛应用程序提供当前和未来数据中心的性能保证和资源利用方面的显著改进。此外，所提出的方法促进了指导鲁棒复杂系统设计的基础分析，并可能在一般情况下为数据中心和云计算产生新的理论。通过现有的课程设置和REU计划，少数民族和女性学生以及残疾学生参与这项拟议的研究将提高他们在未来就业市场的竞争力。

The ultimate design goal of datacenter services is to enable user-centric services that meet diverse Quality-of-Experience (QoE) requirements for individual users, while allowing for optimized and fair computing and networking resource sharing among users. However, achieving this goal is by no means easy. Existing datacenter job scheduling and resource provisioning mechanisms are bottom-up, point solutions, addressing partial aspects of the related design challenges only, and hence, fail to achieve the above goal.       The proposed research will develop a top-down, holistic approach concerning the entire datacenter systems stack, aiming at achieving the dual goal. It starts from the very top, i.e., the QoE or user utility requirements at the user application level. It then works its way downward, by mapping user-level requirements into datacenter system-level resource demands, which, together with the datacenter computing and networking resource constraints, set the boundary conditions for a proposed user-utility-aware optimization framework. Finally, an example software-based design and implementation of the proposed solution will be developed for a state-of-the-art, distributed datacenter. Successfully achieving this goal will provide significant improvement in performance guarantees and resource utilization in current and future datacenters for a wide range of applications with diverse QoE requirements. Moreover, the proposed approach promotes fundamental analysis in guiding the design of robust complex systems and may give rise to new theories for datacenter and cloud computing in general. The involvement of underrepresented minority and women students, and students with disability in this proposed research through existing course offerings and a REU program will enhance their competitiveness in the future job market.