AF: Medium: Collaborative Research: Multi-dimensional Scheduling and Resource Allocation in Data Centers

数据中心和集群计算平台已成为过去十年的主流计算模式，成为执行大数据工作负载的事实上的方法。通常，整个集群被视为由提交需要不同类型资源的作业的多个客户端共享的一组资源。为了增加资源需求的异质性(或维度)，集群中的机器在它们提供的资源方面可以是异构性的。在这种情况下，需要对资源进行适当的分配和定价，以平衡性能和需求。在这个项目中，PI试图研究数据中心的作业调度，以优化时间服务质量指标，如响应时间和公平性，当作业可能具有多维的资源需求时。研究的主要问题可以表述为：作业调度的时间性质如何与资源需求的维度相互作用，以及这两者又如何与激励和公平的经典经济愿望相互作用？本项目与以前的工作不同，旨在通过理论计算机科学的视角开发适当的模型和算法，特别是通过融合近似和在线算法、算法博弈论和随机优化等不同领域。由此得到的见解还将被用来开发新的技术，以解决经典的调度和博弈论问题，这些问题已经没有成功的解决方案。该项目是跨学科的，开发的理论模型和技术将受到新兴技术产生的新硬件架构的应用领域以及在数据中心内配置它们所产生的异构性的推动。此外，将通过模拟数据中心执行的痕迹以及在集群上进行部署和实验来执行经验验证。这最终将影响使用和生成海量数据的互联网系统的设计和部署。该项目的跨学科性质不仅表明，需要培训从高中生到毕业生的学生管道，向他们传授算法思维的力量及其更广泛的相关性，而且还需要将科学家、数学家和系统构建者聚集到同一个平台上，积极交流思想。为此，私人投资机构力求通过包括有效教学和指导的教育计划，使包括妇女和少数群体在内的下一代学生掌握相关的算法技能，并通过组织讲习班和撰写书籍和调查，广泛传播拟议的工作。

Data centers and cluster computing platforms have become the dominant computational paradigms of the past decade, becoming the de facto method of executing Big Data workloads. Typically, the entire cluster is treated as a set of resources shared by multiple clients who submit jobs requiring different types of resources. To add to this heterogeneity (or dimensionality) in resource requirements, machines in a cluster can be heterogeneous in terms of resources they provide. In such settings, resources need to be allocated and priced appropriately so as to balance performance with demand.In this project, the PIs seek to study job scheduling in data centers to optimize temporal Quality of Service metrics such as response time along with fairness, when jobs can have resource requirements that are multi-dimensional. The main question studied can be phrased as: How does the temporal nature of job scheduling interplay with the dimensionality of resource requirements, and how do these two in turn interact with the classical economic desiderata of incentives and fairness?This project is differentiated from previous work in aiming to develop appropriate models and algorithms through the lens of theoretical computer science, particularly by a fusion of the disparate fields of approximation and online algorithms, algorithmic game theory, and stochastic optimization. The resulting insights will be used to also develop new techniques to address classical scheduling and game theoretic problems that have defied successful solutions. The project is interdisciplinary, and the theoretical models and techniques developed will be motivated by the application domain of new hardware architectures stemming from emerging technologies, and the heterogeneity arising from provisioning them within a data center. Further, empirical validations will be performed, both via simulation on traces from data center executions, as well as deployment and experiments on clusters. This will ultimately influence the design and deployment of internet systems that use and generate massive data.The interdisciplinary nature of the project points to not only the need for training a pipeline of students from high school students to graduates and imparting to them the power of algorithmic thinking and its broader relevance, but also the necessity for bringing scientists, mathematicians, and system builders to the same platform for active exchange of ideas. Towards this end, the PIs seek to equip the next generation of students, including women and minorities, with the relevant algorithmic skills by an education plan that includes effective teaching and mentorship, as well as to broadly disseminate the proposed work by organizing workshops and by writing books and surveys.