CSR: Medium: Pythia: An Application Analysis and Online Modeling Based Prediction Framework for Scalable Resource Management

在高速物理、经济学、基因组学、天文学和气象学等各种科学领域，处理大量信息的计算机应用程序正变得越来越普遍。该项目的总体目标是设计软件工具和技术，以在高级计算系统上有效地支持此类应用程序。此外，用于实现这种先进系统的硬件通常具有不同类型的资源，例如，与专用图形处理单元一起运行的常规计算机处理器，并且当以所需的大规模运行应用程序时，这种异构性提出了主要挑战。更好地了解新兴硬件上的应用程序行为是维持这些系统的关键。为此，该项目设计和开发了Pythia，这是一种对应用程序在给定硬件上的行为进行建模和预测的软件。这些信息被用来更好地利用资源，实现可扩展的高性能计算系统。本研究的智力价值涉及三个中间研究目标。1)使用编译时程序分析设计准确的应用程序分类器，捕获工作流行为和应用程序特征，并提供对预期运行时应用程序交互的详细洞察。2)设计和开发一个准确的模拟模型，该模型将工作流和应用程序的特征结合到启发式引擎中，以预测应用程序在给定条件和资源下的执行情况。3)设计一个分布式的、灵活的、高效的、易于使用的在线Oracle框架，该框架捕获基础架构的异构性，并与实时系统集成以预测应用程序的行为，进而帮助指导针对应用程序的资源调度和管理。该项目的完成将为实现更高效和可扩展的计算系统创造工具和技术。这项工作影响了经常使用高性能大规模计算系统的广泛学科，特别是在数据驱动的发现方面。因此，使用Pythia将缩短现代和新兴应用程序的解决方案时间，从而直接影响我们的生活方式。这些教育活动包括招募和指导妇女和少数民族学生，将有助于培养出具有高度市场技能的毕业生。将开源并公开的研究发现和软件工具整合到教育课程中，将有助于吸引下一代计算机科学家的兴趣。

Computer applications that process large amounts of information are becoming common in a variety of science domains, such as High-Speed Physics, Economics, Genomics, Astronomy, and Meteorology. The overall goal of this project is to design software tools and technologies to support such applications efficiently on advanced computing systems. Moreover, the hardware that is used to implement such advanced systems often boasts of different types of resources, e.g., a conventional computer processor running alongside specialized graphic processing units, and this heterogeneity presents a major challenge when running the applications at the needed large scale. Having a better understanding of the applications behavior on the emerging hardware is key to sustaining these systems. To this end, the project designs and develops Pythia, software that models and predicts how applications would behave on given hardware. This information is then used to better utilize the resources, and achieve scalable and high performance computing systems.The intellectual value of this research involves three intermediate research goals. 1) Design an accurate application classifier using compile-time program analysis that captures workflow behavior and application characteristics, and provides detailed insights into expected runtime application interactions. 2) Design and develop an accurate simulation model that incorporates workflow and application characteristics into a heuristics engine to predict how the application will perform under given conditions and resources. 3) Design a distributed, flexible, efficient, and easy-to-use online oracle framework that captures the infrastructure heterogeneity and integrates with live systems to predict application behavior, which in turn can help guide application-attuned resource scheduling and management. Completion of the project will create tools and technologies for realization of more efficient and scalable computing systems. This work impacts a broad range of disciplines that regularly employ high-performance large-scale computing systems, especially for data-driven discovery. Consequently, use of Pythia will reduce the time-to-solution for modern and emerging applications, and therefore directly affect our way of life. The educational activities, which include recruiting and mentoring women and minority students, will help produce graduates with highly marketable skill sets. The integration of the research discoveries and software tools, which will be open source and made public, into the educational curriculum will help capture the interest of the next generation of computer scientists.