CSR: Small: XooMR: Cross-Layer and Cross-Phase Cooperation for Fair and Efficient MapReduce

MapReduce作为一种新的编程模型被广泛利用，以挖掘大数据并行数据处理的能力。越来越多的系统被部署到用MapReduce编写的数据密集型分析应用程序中。许多作业可能同时出现在资源需求冲突的系统上。该项目研究了实现系统效率的跨层协作技术，提高作业公平性和系统吞吐量的跨阶段技术，以及利用作业之间的时间关系提高吞吐量和服务于由多个MapReduce作业组成的分析查询的跨作业任务协同调度技术。这个项目在几个方面产生了深远的影响。这些措施包括：(1)加强奥本大学的计算机科学课程，并通过MapReduce学生研究项目提高教学效率；(2)招募和培养不同背景的学生，特别是少数族裔和女性学生群体从事计算机职业；(3)以出版物、演讲、会议教程、演示等形式传播研究成果，发布开源软件代码，最终推动其整合到Hadoop官方代码库中；(4)与产业界合作，加强研究伙伴关系，培育技术向产业界转移的机会。

MapReduce has been widely leveraged as a new programming model to tap the power of parallel data processing for Big Data. More and more systems are being deployed to serve data-intensive analytics applications written in MapReduce. Many jobs can show up at the same time on a system with conflicting resource requirements. This project investigates cross-layer cooperation techniques to achieve system efficiency, cross-phase techniques to enhance job fairness and system throughput, and cross-job task co-scheduling techniques to exploit the temporal relationship among jobs for better throughput and services to analytic queries composed of multiple MapReduce jobs.This project has profound impacts in several aspects. These include (1) strengthening computer science courses at Auburn University, and enhancing instruction effectiveness with student research projects on MapReduce; (2) recruiting and cultivating students of diverse backgrounds, particularly under-represented minority and female student groups for careers in computing; (3) disseminating research results as publications, presentations, conference tutorials and demonstrations, releasing open-source software codes, and eventually pushing them for integration into the official Hadoop code base; and (4) collaborating with industry, strengthening research partnerships, and cultivating opportunities for technology transfer to industry.