CSR: Medium: Pythia: An Application Analysis and Online Modeling Based Prediction Framework for Scalable Resource Management

CSR：中：Pythia：基于应用分析和在线建模的可扩展资源管理预测框架

• 批准号: 1405697
• 负责人: Ali Butt
• 金额: $ 75万
• 依托单位: Virginia Polytechnic Institute and State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-10-01 至 2020-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1405697&HistoricalAwards=false
• 关键词: CSR; Medium; Pythia; Application; Analysis

Computer applications that process large amounts of information are becoming common in a variety of science domains, such as High-Speed Physics, Economics, Genomics, Astronomy, and Meteorology. The overall goal of this project is to design software tools and technologies to support such applications efficiently on advanced computing systems. Moreover, the hardware that is used to implement such advanced systems often boasts of different types of resources, e.g., a conventional computer processor running alongside specialized graphic processing units, and this heterogeneity presents a major challenge when running the applications at the needed large scale. Having a better understanding of the applications behavior on the emerging hardware is key to sustaining these systems. To this end, the project designs and develops Pythia, software that models and predicts how applications would behave on given hardware. This information is then used to better utilize the resources, and achieve scalable and high performance computing systems.The intellectual value of this research involves three intermediate research goals. 1) Design an accurate application classifier using compile-time program analysis that captures workflow behavior and application characteristics, and provides detailed insights into expected runtime application interactions. 2) Design and develop an accurate simulation model that incorporates workflow and application characteristics into a heuristics engine to predict how the application will perform under given conditions and resources. 3) Design a distributed, flexible, efficient, and easy-to-use online oracle framework that captures the infrastructure heterogeneity and integrates with live systems to predict application behavior, which in turn can help guide application-attuned resource scheduling and management. Completion of the project will create tools and technologies for realization of more efficient and scalable computing systems. This work impacts a broad range of disciplines that regularly employ high-performance large-scale computing systems, especially for data-driven discovery. Consequently, use of Pythia will reduce the time-to-solution for modern and emerging applications, and therefore directly affect our way of life. The educational activities, which include recruiting and mentoring women and minority students, will help produce graduates with highly marketable skill sets. The integration of the research discoveries and software tools, which will be open source and made public, into the educational curriculum will help capture the interest of the next generation of computer scientists.

处理大量信息的计算机应用程序在各种科学领域中变得越来越普遍，例如高速物理学，经济学，基因组学，天文学和气象学。该项目的总体目标是设计软件工具和技术，以支持这些应用程序在先进的计算系统有效。此外，用于实现这种高级系统的硬件通常拥有不同类型的资源，例如，传统的计算机处理器与专用图形处理单元一起运行，并且当以所需的大规模运行应用程序时，这种异构性提出了主要的挑战。更好地理解新兴硬件上的应用程序行为是维持这些系统的关键。为此，该项目设计和开发了Pythia，这是一种软件，可以建模和预测应用程序在给定硬件上的行为。这些信息将被用来更好地利用资源，并实现可扩展的和高性能的计算系统。1)使用编译时程序分析设计一个精确的应用程序分类器，该分析捕获工作流行为和应用程序特征，并提供对预期运行时应用程序交互的详细见解。2)设计和开发一个精确的仿真模型，将工作流和应用程序特性整合到一个仿真引擎中，以预测应用程序在给定条件和资源下的性能。3)设计一个分布式的、灵活的、高效的、易于使用的在线Oracle框架，它可以捕获基础设施的异构性，并与实时系统集成以预测应用程序的行为，从而帮助指导与应用程序相协调的资源调度和管理。该项目的完成将为实现更高效和可扩展的计算系统创造工具和技术。这项工作影响了经常使用高性能大规模计算系统的广泛学科，特别是对于数据驱动的发现。因此，Pythia的使用将减少现代和新兴应用的解决方案的时间，从而直接影响我们的生活方式。教育活动包括招聘和指导妇女和少数民族学生，将有助于培养具有高度市场化技能的毕业生。将研究发现和软件工具（将开放源码并公开）纳入教育课程将有助于吸引下一代计算机科学家的兴趣。

项目成果

iez: Resource Contention Aware Load Balancing for Large-Scale Parallel File Systems

• DOI: 10.1109/ipdps.2019.00070
• 发表时间: 2019-05
• 期刊: 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
• 作者: Bharti Wadhwa;A. Paul;Sarah Neuwirth;Feiyi Wang;S. Oral;A. Butt;Jon Bernard;K. Cameron

Toward Transparent Data Management in Multi-Layer Storage Hierarchy of HPC Systems

• DOI: 10.1109/ic2e.2018.00046
• 发表时间: 2018-05
• 期刊: 2018 IEEE International Conference on Cloud Engineering (IC2E)
• 作者: Bharti Wadhwa;S. Byna;A. Butt

Toward scalable monitoring on large-scale storage for software defined cyberinfrastructure

• DOI: 10.1145/3149393.3149402
• 发表时间: 2017-11
• 期刊: Proceedings of the 2nd Joint International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems
• 作者: A. Paul;S. Tuecke;Ryan Chard;A. Butt;K. Chard;Ian T Foster

CSLIM: Automated Extraction of IoT Functionalities from Legacy C Codebases

CSLIM：从遗留 C 代码库中自动提取 IoT 功能

• 发表时间: 2019
• 期刊: Proceedings of the 8th International Workshop on Computing and Networking for IoT and Beyond (ComNet-IoT '19
• 作者: Sim, Hyogi;Paul, Arnab K.;Tilevich, Eli;Butt, Ali R.;Shahzad, Muhammad
• 通讯作者: Shahzad, Muhammad

Improving I/O Performance of HPC Application Using Intra-Job Scheduling.

使用作业内调度提高 HPC 应用程序的 I/O 性能。

• 发表时间: 2019
• 期刊: Proceedings of the 4th Joint International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems (PDSW-DISC'19
• 作者: Paul, Arnab K.';Moody, Adam;Gonsiorowski, Elsa;Mohror, Kathryn;Butt, Ali R.
• 通讯作者: Butt, Ali R.