Application Characterization for Adaptive Computing Platform Determination for Computational and Data-Enabled Science and Engineering

传统的高性能计算集群(HPCC)和Hadoop集群都是计算和数据科学与工程(CDS&amp；E)的重要平台。Hadoop MapReduce在大规模数据分析中通常是有效的，而传统的HPC更常用于计算问题(PB与Petaflop)。HPCC可以包括填充层，该填充层允许Hadoop MapReducte访问HPC存储(由Hadoop+HPCC表示)、本地存储(由Hadoop表示)以及两者的组合(由Hadoop/HPCC表示)。该项目旨在识别各种CDS&amp；E MapReduce计算任务的特点，并根据它们的特点自适应地确定适合个别应用程序的最佳平台(Hadoop、Hadoop+HPCC和Hadoop/HPCC)，并在给定性能目标和系统成本指标的情况下，在本地存储和专用远程存储之间优化安排数据放置。更广泛的影响包括对不同计算平台是否适合不同CDS&A；E应用的关键洞察，以及更先进的HPC系统。研究成果将通过向行业合作伙伴转让技术、在同行评议期刊上发表以及在软件发布中传播。研究成果还将成为网络基础设施研究的催化剂，网络基础设施为CDS&E领域提供服务。该项目将为参与的克莱姆森大学毕业生、本科生、教师和K-12学生提供全面的学生培训和合作研究机会。成绩会纳入督导主任所教授的课程。专业督学将招收新学生，特别是那些来自代表性不足的群体的学生，以攻读STEM学科。

Traditional high performance computing clusters (HPCC) and Hadoop clusters are both important platforms in computational and data-enabled science and engineering (CDS&E). Hadoop MapReduce is typically effective in large-scale data analysis while traditional HPC is more commonly employed in computational problems (petabytes vs. petaflops). HPCC can include a shim layer that allows Hadoop MapReduce to access HPC storage (denoted by Hadoop+HPCC), local storage (denoted by Hadoop), and a combination of both (denoted by Hadoop/HPCC). This project aims to identify the characteristics of various CDS&E MapReduce computational tasks, and adaptively determine the best platform (Hadoop, Hadoop+HPCC and Hadoop/HPCC) for individual applications based on their characteristics, and also optimally arrange data placement between local storage and dedicated remote storage, given performance objectives and system cost metrics. Broader impacts include critical insights into the suitability of different computing platforms to different CDS&E applications, and a more advanced HPC system. Research results will be disseminated through technology transfer to industry partners, via publication in peer review journals, and in software releases. Results will also serve as catalyst for research in cyberinfrastructure, which serves the CDS&E fields. This project will provide thorough training of students and collaborative research opportunities for participating Clemson graduates, undergraduates, faculty, and K-12 students. Results will be integrated into courses taught by the PIs. The PIs will recruit new students, particularly those from underrepresented groups, to undertake the study of a STEM discipline.