Application Characterization for Adaptive Computing Platform Determination for Computational and Data-Enabled Science and Engineering

传统的高性能计算集群（HPCC）和Hadoop集群都是计算和数据支持的科学与工程（CDS E）中的重要平台。Hadoop MapReduce通常在大规模数据分析中有效，而传统HPC更常用于计算问题（PB与Petaflops）。HPCC可以包括允许Hadoop MapReduce访问HPC存储（由Hadoop+HPCC表示）、本地存储（由Hadoop表示）以及两者的组合（由Hadoop/HPCC表示）的垫片层。该项目旨在识别各种CDS E MapReduce计算任务的特征，并根据其特征自适应地确定单个应用程序的最佳平台（Hadoop，Hadoop+HPCC和Hadoop/HPCC），并在给定性能目标和系统成本指标的情况下，优化本地存储和专用远程存储之间的数据放置。更广泛的影响包括对不同计算平台适用于不同CDS E应用程序的重要见解，以及更先进的HPC系统。研究成果将通过向行业伙伴转让技术、在同行评审期刊上发表和发布软件等方式传播。结果也将作为催化剂的研究网络基础设施，这服务于CDS E领域。该项目将为参与的克莱姆森毕业生、本科生、教师和K-12学生提供全面的学生培训和合作研究机会。研究结果将纳入方案研究员教授的课程。PI将招募新的学生，特别是那些来自代表性不足的群体，进行STEM学科的研究。

Traditional high performance computing clusters (HPCC) and Hadoop clusters are both important platforms in computational and data-enabled science and engineering (CDS&E). Hadoop MapReduce is typically effective in large-scale data analysis while traditional HPC is more commonly employed in computational problems (petabytes vs. petaflops). HPCC can include a shim layer that allows Hadoop MapReduce to access HPC storage (denoted by Hadoop+HPCC), local storage (denoted by Hadoop), and a combination of both (denoted by Hadoop/HPCC). This project aims to identify the characteristics of various CDS&E MapReduce computational tasks, and adaptively determine the best platform (Hadoop, Hadoop+HPCC and Hadoop/HPCC) for individual applications based on their characteristics, and also optimally arrange data placement between local storage and dedicated remote storage, given performance objectives and system cost metrics. Broader impacts include critical insights into the suitability of different computing platforms to different CDS&E applications, and a more advanced HPC system. Research results will be disseminated through technology transfer to industry partners, via publication in peer review journals, and in software releases. Results will also serve as catalyst for research in cyberinfrastructure, which serves the CDS&E fields. This project will provide thorough training of students and collaborative research opportunities for participating Clemson graduates, undergraduates, faculty, and K-12 students. Results will be integrated into courses taught by the PIs. The PIs will recruit new students, particularly those from underrepresented groups, to undertake the study of a STEM discipline.