EAGER: Tadoop: A Dual-Purpose Framework Taming the Bipolarity of Storage and Communication for High-Performance Computing and Data Analytics

高性能计算(HPC)提供商和应用程序需要下一代解决方案来处理来自科学模拟的大数据。国家实验室和大学中的传统HPC系统是基于以计算为中心的范例构建的，而企业大数据分析应用程序更喜欢以数据为中心的范例，如MapReduce.这两个范例之间截然不同的体系结构差异要求采用非常规方法。该项目采用了一种截然不同的方法来研究以计算为中心和以数据为中心的范例中的关键体系结构组件，设计了名为Tadoop的变革性双用途框架，该框架解决了存储和通信管理中的两极问题，并将它们统一用于HPC和企业分析应用程序。这个高风险的Tadoop框架可以为高性能计算和数据分析应用程序带来变革性的数据基础设施，并在几个方面产生更广泛的影响，例如展示现有的高性能计算基础设施转变为计算和分析两用系统，改善计算机科学课程和教学有效性，加强多学科数据分析研究，发布开源软件代码，以及将技术转移为商业服务。

High-performance computing (HPC) providers and applications need next-generation solutions to process big data from scientific simulations. Conventional HPC systems found in national laboratories and universities are constructed based on the compute-centric paradigm while enterprise big data analytics applications prefer a data-centric paradigm such as MapReduce. Distinct architectural differences between these two paradigms demand unconventional approaches. This project takes a radically different approach to investigate key architectural components in compute-centric and data-centric paradigms, designs a transformative dual-purpose framework called Tadoop that addresses their bipolarity issues in storage and communication management, and unifies them for both HPC and enterprise analytics applications. This high-risk Tadoop framework can enable a transformative data infrastructure for both HPC and data analytics applications and lead to broader impact in several aspects, such as demonstrating the transformation of existing HPC infrastructures into dual-purpose systems for computing and analytics, improving computer science curricula and instruction effectiveness, strengthening multidisciplinary data analytics research, releasing open-source software code, and transferring technologies for commercial service.