Application Characterization for Adaptive Computing Platform Determination for Computational and Data-Enabled Science and Engineering

传统的高性能计算集群（HPCC）和Hadoop集群都是计算和数据支持科学与工程（CDS&amp；E）的重要平台。Hadoop MapReduce在大规模数据分析中通常是有效的，而传统的HPC更常用于计算问题（pb和petaflops）。HPCC可以包括一个sim层，它允许Hadoop MapReduce访问HPC存储（用Hadoop+HPCC表示），本地存储（用Hadoop表示），以及两者的组合（用Hadoop/HPCC表示）。本项目旨在确定各种CDS&amp；E MapReduce计算任务的特征，并根据其特征自适应地确定单个应用程序的最佳平台（Hadoop， Hadoop+HPCC和Hadoop/HPCC），并在给定性能目标和系统成本指标的情况下，在本地存储和专用远程存储之间最佳地安排数据放置。更广泛的影响包括对不同计算平台对不同CDS&amp；E应用程序的适用性的关键见解，以及更先进的HPC系统。研究成果将通过向工业伙伴转让技术、在同行评审期刊上发表和在软件发布中传播。研究结果还将成为网络基础设施研究的催化剂，为cd&e领域服务。该项目将为参与克莱姆森大学的毕业生、本科生、教师和K-12学生提供全面的学生培训和合作研究机会。结果将被整合到pi教授的课程中。pi将招收新学生，特别是那些来自代表性不足的群体的学生，从事STEM学科的研究。

Traditional high performance computing clusters (HPCC) and Hadoop clusters are both important platforms in computational and data-enabled science and engineering (CDS&E). Hadoop MapReduce is typically effective in large-scale data analysis while traditional HPC is more commonly employed in computational problems (petabytes vs. petaflops). HPCC can include a shim layer that allows Hadoop MapReduce to access HPC storage (denoted by Hadoop+HPCC), local storage (denoted by Hadoop), and a combination of both (denoted by Hadoop/HPCC). This project aims to identify the characteristics of various CDS&E MapReduce computational tasks, and adaptively determine the best platform (Hadoop, Hadoop+HPCC and Hadoop/HPCC) for individual applications based on their characteristics, and also optimally arrange data placement between local storage and dedicated remote storage, given performance objectives and system cost metrics. Broader impacts include critical insights into the suitability of different computing platforms to different CDS&E applications, and a more advanced HPC system. Research results will be disseminated through technology transfer to industry partners, via publication in peer review journals, and in software releases. Results will also serve as catalyst for research in cyberinfrastructure, which serves the CDS&E fields. This project will provide thorough training of students and collaborative research opportunities for participating Clemson graduates, undergraduates, faculty, and K-12 students. Results will be integrated into courses taught by the PIs. The PIs will recruit new students, particularly those from underrepresented groups, to undertake the study of a STEM discipline.