CCRI: ENS: Collaborative Research: Open Computer System Usage Repository and Analytics Engine

CCRI：ENS：协作研究：开放计算机系统使用存储库和分析引擎

• 批准号: 2016704
• 负责人: Saurabh Bagchi
• 金额: $ 118.39万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2016704&HistoricalAwards=false
• 关键词: CCRI; ENS; Collaborative; Research; Open

In science and engineering research, large-scale, centrally managed computing clusters or “supercomputers” have been instrumental in enabling the kinds of resource-intensive simulations, analyses, and visualizations that have been used in computer-aided drug discovery, high strength materials design for cars and jet engines, and disease vector analysis to name a few. Such clusters are complex systems comprised of several hundred to thousand computer servers with fast network connections between them, various data storage resources, and highly optimized scientific software being shared with several hundred other researchers from diverse domains. Consequently, the overall dependability of such systems relies on the dependability of these individual highly interconnected elements as well as the characteristics of cascading failures. While computer systems researchers and practitioners have been at the forefront of designing and deploying dependable computing cluster systems, this task has been hampered by the lack of publicly available, real-world failure data from supercomputers currently in operation. Prior practice has largely involved tedious, manual collection and curation of small sets of data for use in specific analyses. This project will establish seamless, automated pipelines for acquiring, processing, and curating continuous, detailed system usage, monitoring, and failure data from large computing clusters at two organizations, Purdue University and the University of Texas at Austin. This data will be disseminated through a publicly accessible portal and complemented by a suite of in-situ analytics capabilities that will support and spur research in dependable computing systems. The data acquisition pipeline and analytics software will be made open-source and designed for ease of federation, extension, and adoption to cluster systems operated by other organizations.Cluster computing systems are a key resource in time-sensitive, computationally intensive research such as virus structure modeling and drug discovery and have been at the forefront of efforts to tackle global pandemics. Both unanticipated system down-times and lack of actionable feedback to researchers on computational failures can have adverse effects on research timeliness and efficiency. This project will allow the practitioners and administrators of these systems to develop data-backed best practices for ensuring high availability and utilization for their clusters. The resulting large, public data repository consisting of data from clusters with diverse workloads spanning traditional high-performance computing, modern accelerator-based computing (for example on graphics processing units (GPUs)), and cloud-style applications will allow the systems research community to consider forward-looking research questions based on real system data. The project will train a cadre of students in data analysis on live production systems and this will provide them with a unique learning experience, interfacing with a variety of stakeholders.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在科学和工程研究中，大规模、集中管理的计算集群或“超级计算机”已经有助于实现各种资源密集型模拟、分析和可视化，所述资源密集型模拟、分析和可视化已经用于计算机辅助药物发现、用于汽车和喷气发动机的高强度材料设计以及疾病媒介分析，仅举几例。这些集群是由数百到数千台计算机服务器组成的复杂系统，它们之间具有快速的网络连接，各种数据存储资源，以及与来自不同领域的数百名其他研究人员共享的高度优化的科学软件。因此，这种系统的整体可靠性依赖于这些独立的高度互连的元件的可靠性以及级联故障的特性。虽然计算机系统研究人员和从业人员一直处于设计和部署可靠的计算集群系统的最前沿，但由于缺乏来自当前运行的超级计算机的公开可用的真实世界故障数据，这项任务一直受到阻碍。先前的实践主要涉及繁琐的手动收集和管理用于特定分析的小数据集。该项目将建立无缝的自动化管道，用于获取，处理和管理来自普渡大学和德克萨斯大学奥斯汀分校两个组织的大型计算集群的连续，详细的系统使用，监控和故障数据。这些数据将通过一个可公开访问的门户网站传播，并辅之以一套现场分析能力，以支持和促进可靠计算系统的研究。数据采集管道和分析软件将开放源代码，旨在方便联合、扩展和采用其他组织运营的集群系统。集群计算系统是时间敏感、计算密集型研究（如病毒结构建模和药物发现）的关键资源，并一直处于应对全球流行病的最前沿。意外的系统停机时间和缺乏可操作的反馈给研究人员的计算故障可能会对研究的及时性和效率产生不利影响。该项目将使这些系统的从业人员和管理人员能够制定以数据为后盾的最佳做法，以确保其集群的高可用性和利用率。由此产生的大型公共数据存储库由来自集群的数据组成，这些集群具有跨越传统高性能计算、现代基于加速器的计算（例如图形处理单元（GPU））和云式应用程序的各种工作负载，这将使系统研究社区能够考虑基于真实的系统数据的前瞻性研究问题。该项目将培训一批学生进行现场生产系统的数据分析，这将为他们提供独特的学习体验，与各种利益相关者进行交流。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

ORION and the Three Rights: Sizing, Bundling, and Prewarming for Serverless DAGs

ORION 和三项权利：无服务器 DAG 的规模调整、捆绑和预热

• 发表时间: 2022
• 期刊: 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22
• 作者: Mahgoub, Ashraf;Yi, Edgardo Barsallo;Shankar, Karthick;Elnikety, Sameh;Chaterji, Somali;Bagchi, Saurabh
• 通讯作者: Bagchi, Saurabh

Root Cause Analysis of Failures in Microservices through Causal Discovery

• 发表时间: 2022
• 作者: Azam Ikram;Sarthak Chakraborty;Subrata Mitra;S. Saini;S. Bagchi;Murat Kocaoglu

SONIC: Application-aware Data Passing for Chained Serverless Applications

• 发表时间: 2021
• 作者: Ashraf Y. Mahgoub;K. Shankar;S. Mitra;Ana Klimovic;S. Chaterji;S. Bagchi

Closing-the-Loop: A Data-Driven Framework for Effective Video Summarization

• DOI: 10.1109/ism.2020.00042
• 发表时间: 2020-12
• 期刊: 2020 IEEE International Symposium on Multimedia (ISM)
• 作者: Ran Xu;Haoliang Wang;Stefano Petrangeli;Viswanathan Swaminathan;S. Bagchi

An Automated Approach to Re-Hosting Embedded Firmware Through Removing Hardware Dependencies.

通过删除硬件依赖性来重新托管嵌入式固件的自动化方法。

• DOI: 10.2172/2006057
• 发表时间: 2022
• 期刊: IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW
• 作者: Ketterer, Austin;Shekar, Asha;Yi, Edgardo;Bagchi, Saurabh;Clements, Abraham
• 通讯作者: Clements, Abraham