Hadoop YARN已经成为微软，腾讯等互联网公司部署大数据计算集群所采用的主流计算平台。在Hadoop YARN中，掌控好资源分配就抓住了整个平台的核心。现有的Hadoop YARN平台下的资源分配机制主要考虑的是不同租户之间的公平性问题，并且都是基于启发式算法设计的。这些机制缺乏系统的分析和论证，在作业执行时间和资源使用率这些性能指标上存在着很大的提升空间。本项目将设计一种基于在线优化理论的Hadoop YARN平台下的资源分配机制来保证租户公平性、提升系统容错能力、减小作业延迟。为了达到此目标，本项目将建立一个多维资源约束下的作业响应时间最优化模型，利用所熟悉的在线优化理论工具设计算法并且对提出的算法进行严格的理论分析。另外，本项目也会将设计的算法实现在Hadoop YARN开源平台下，并采用仿真与真实实验相结合的办法对所设计算法进行全面评估。

Hadoop YARN has emerged as a major Big Data Processing Platform for many IT companies including Microsoft and Tencent. Job Scheduling and Resource Allocation for Hadoop YARN are essential to the overall system performance. However, most existing resource allocation schemes under Hadoop mainly concern about the fairness between different users/applications and are designed based on simple heuristics. There is a lack of systematic analysis or reference study on how well or suboptimal these schemes are when comparing to the theoretical limits of various performance metrics such as overall job response time and system utilization. To tackle this issue, this project will redesign resource allocation algorithms from an optimization perspective. In particular, we will build a system model based on the Multi-dimensional Bin Packing problem, taking job response time, fairness and fault tolerance into consideration so as to optimize the overall job flowtime. We will adopt the Dual-fitting framework to solve the optimization problem and design corresponding algorithms. To demonstrate the practicality of our designs, this project will also implement the proposed algorithms as extensions of an open-source software platform, i.e., Apache Hadoop YARN. Moreover, we will compare our proposed solutions with other baseline schemes.