AF: Medium: Collaborative Research: Multi-dimensional Scheduling and Resource Allocation in Data Centers

数据中心和集群计算平台已经成为过去十年的主要计算模式，成为执行大数据工作负载的事实上的方法。通常，整个群集被视为由提交需要不同类型资源的作业的多个客户端共享的一组资源。为了增加资源需求的这种异质性（或维度），集群中的机器在它们提供的资源方面可以是异质的。在这样的环境中，资源需要适当地分配和定价，以平衡性能与需求。在这个项目中，PI寻求研究在数据中心的作业调度，以优化时间服务质量指标，如响应时间沿着与公平性，当作业可以有多维度的资源需求。 研究的主要问题可以表述为：工作调度的时间性质如何与资源需求的维度相互作用，以及这两个又如何与经典的经济激励和公平的必要条件相互作用？这个项目是区别于以往的工作，旨在开发适当的模型和算法，通过理论计算机科学的透镜，特别是通过融合不同领域的近似和在线算法，算法博弈论，随机优化。由此产生的见解也将被用来开发新的技术，以解决经典的调度和博弈论的问题，挑战成功的解决方案。 该项目是跨学科的，开发的理论模型和技术将受到新兴技术产生的新硬件架构的应用领域以及在数据中心内提供它们所产生的异构性的推动。此外，还将通过对数据中心执行的跟踪进行模拟以及对集群进行部署和实验来进行经验验证。 这将最终影响使用和生成海量数据的互联网系统的设计和部署。该项目的跨学科性质表明，不仅需要培训从高中生到研究生的学生管道，并向他们传授算法思维的力量及其更广泛的相关性，而且还需要将科学家，数学家，和系统建设者在同一平台上积极交流思想。为此目的，项目执行人力求通过一项包括有效教学和辅导在内的教育计划，使下一代学生，包括妇女和少数民族，掌握相关的算法技能，并通过组织讲习班、写书和调查，广泛宣传拟议的工作。

Data centers and cluster computing platforms have become the dominant computational paradigms of the past decade, becoming the de facto method of executing Big Data workloads. Typically, the entire cluster is treated as a set of resources shared by multiple clients who submit jobs requiring different types of resources. To add to this heterogeneity (or dimensionality) in resource requirements, machines in a cluster can be heterogeneous in terms of resources they provide. In such settings, resources need to be allocated and priced appropriately so as to balance performance with demand.In this project, the PIs seek to study job scheduling in data centers to optimize temporal Quality of Service metrics such as response time along with fairness, when jobs can have resource requirements that are multi-dimensional. The main question studied can be phrased as: How does the temporal nature of job scheduling interplay with the dimensionality of resource requirements, and how do these two in turn interact with the classical economic desiderata of incentives and fairness?This project is differentiated from previous work in aiming to develop appropriate models and algorithms through the lens of theoretical computer science, particularly by a fusion of the disparate fields of approximation and online algorithms, algorithmic game theory, and stochastic optimization. The resulting insights will be used to also develop new techniques to address classical scheduling and game theoretic problems that have defied successful solutions. The project is interdisciplinary, and the theoretical models and techniques developed will be motivated by the application domain of new hardware architectures stemming from emerging technologies, and the heterogeneity arising from provisioning them within a data center. Further, empirical validations will be performed, both via simulation on traces from data center executions, as well as deployment and experiments on clusters. This will ultimately influence the design and deployment of internet systems that use and generate massive data.The interdisciplinary nature of the project points to not only the need for training a pipeline of students from high school students to graduates and imparting to them the power of algorithmic thinking and its broader relevance, but also the necessity for bringing scientists, mathematicians, and system builders to the same platform for active exchange of ideas. Towards this end, the PIs seek to equip the next generation of students, including women and minorities, with the relevant algorithmic skills by an education plan that includes effective teaching and mentorship, as well as to broadly disseminate the proposed work by organizing workshops and by writing books and surveys.