CSR: Small: XooMR: Cross-Layer and Cross-Phase Cooperation for Fair and Efficient MapReduce

CSR：小：XooMR：跨层跨阶段合作实现公平高效的 MapReduce

• 批准号: 1320016
• 负责人: Weikuan Yu
• 金额: $ 47.5万
• 依托单位: Auburn University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-10-01 至 2015-10-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1320016&HistoricalAwards=false
• 关键词: CSR; Small; XooMR; Cross; Layer

MapReduce has been widely leveraged as a new programming model to tap the power of parallel data processing for Big Data. More and more systems are being deployed to serve data-intensive analytics applications written in MapReduce. Many jobs can show up at the same time on a system with conflicting resource requirements. This project investigates cross-layer cooperation techniques to achieve system efficiency, cross-phase techniques to enhance job fairness and system throughput, and cross-job task co-scheduling techniques to exploit the temporal relationship among jobs for better throughput and services to analytic queries composed of multiple MapReduce jobs.This project has profound impacts in several aspects. These include (1) strengthening computer science courses at Auburn University, and enhancing instruction effectiveness with student research projects on MapReduce; (2) recruiting and cultivating students of diverse backgrounds, particularly under-represented minority and female student groups for careers in computing; (3) disseminating research results as publications, presentations, conference tutorials and demonstrations, releasing open-source software codes, and eventually pushing them for integration into the official Hadoop code base; and (4) collaborating with industry, strengthening research partnerships, and cultivating opportunities for technology transfer to industry.

MapReduce已被广泛用作一种新的编程模型，以挖掘大数据并行数据处理的能力。越来越多的系统被部署来服务于用MapReduce编写的数据密集型分析应用程序。许多作业可能同时出现在具有冲突资源需求的系统上。 该项目研究了实现系统效率的跨层协作技术、提高作业公平性和系统吞吐量的跨阶段技术以及利用作业之间的时间关系以获得更好的吞吐量并为由多个MapReduce作业组成的分析查询提供服务的跨作业任务协同调度技术。该项目在几个方面产生了深远的影响。这些措施包括（1）加强奥本大学的计算机科学课程，并通过学生关于MapReduce的研究项目提高教学效率;（2）招募和培养不同背景的学生，特别是代表性不足的少数民族和女性学生群体从事计算机职业;（三）以出版物、演讲、会议教程和演示等形式传播研究成果，发布开源软件代码，并最终推动它们集成到正式的Hadoop代码库中;（4）与行业合作，加强研究伙伴关系，并培养向行业转移技术的机会。