CI-NEW: Collaborative Research: Computer System Failure Data Repository to Enable Data-Driven Dependability

CI-NEW：协作研究：计算机系统故障数据存储库以实现数据驱动的可靠性

• 批准号: 1513197
• 负责人: Saurabh Bagchi
• 金额: $ 76.33万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-07-01 至 2019-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1513197&HistoricalAwards=false
• 关键词: CI; NEW; Collaborative; Research; Computer

Dependability has become a necessary requisite property for many of the computer systems that surround us or work behind the scenes to support our personal and professional lives. Heroic progress has been made by computer systems researchers and practitioners working together to build and deploy dependable systems. However, an overwhelming majority of this work is not based on real publicly available failure data. Unfortunately, an open failure data repository for any recent computing infrastructure that is large enough, diverse enough and with enough information about the infrastructure and the applications that run on them does not exist. This project will address this pressing need. The research team appreciates that this effort is challenging on many levels. Failure data are considered sensitive and are usually unveiled only before trusting eyes of a small subset of the people at the organization. As part of a current one-year planning grant, this team has collected specific requirements for the repository from a wide audience, collected failure and usage data from the largest centrally managed computing cluster at Purdue and performed preliminary analysis to reveal the workload usage patterns. The goal of this full-scale project is to collect data from a variety of computational infrastructure at the two participating universities, and from several of the NSF-funded large cyberinfrastructure projects.The project will collect, curate, and present public failure data of large-scale computing systems in a repository called FRESCO. The data sets will include static information, dynamic information about the workloads, and failure information for both planned and unplanned outages. The data collection from production machines will have to obey several practical constraints -- no changes to the workload, little performance perturbation, and minimal changes to the operating system. Further, the data have to be sanitized for removing sensitive information and processed to make it interpretable by a broad group of researchers. This project will also provide analysis tools to answer certain commonly occurring questions, such as the correlation between workload and failure and the performance implications of using one library over another, as well as an intuitive graphical front-end which will allow people to explore the data sets and download the relevant ones.Widespread use of the data and the associated analysis tools will give computer systems researchers an unprecedented ability to do data-driven research and offer computing infrastructure providers an analytic-driven capability to run more efficient reliable infrastructures.

对于我们周围或在幕后工作以支持我们的个人和职业生活的许多计算机系统来说，可靠性已成为必要的必备属性。计算机系统研究人员和从业者共同努力构建和部署可靠的系统，取得了巨大的进步。然而，这项工作的绝大多数并不是基于真实的公开故障数据。不幸的是，对于任何最近的计算基础设施来说，不存在一个足够大、足够多样化并且具有关于基础设施和在其上运行的应用程序的足够信息的开放故障数据存储库。该项目将解决这一迫切需求。研究团队认识到这项工作在很多层面上都具有挑战性。故障数据被认为是敏感数据，通常仅在组织中一小部分人员信任的情况下才会公开。作为当前一年计划拨款的一部分，该团队从广泛的受众那里收集了存储库的具体要求，从普渡大学最大的集中管理计算集群收集了故障和使用数据，并进行了初步分析以揭示工作负载使用模式。这个全面项目的目标是从两所参与大学的各种计算基础设施以及 NSF 资助的几个大型网络基础设施项目收集数据。该项目将在名为 FRESCO 的存储库中收集、整理和呈现大型计算系统的公共故障数据。数据集将包括静态信息、有关工作负载的动态信息以及计划内和计划外停机的故障信息。从生产机器收集的数据必须遵守几个实际限制——工作负载不发生变化、性能扰动很小、操作系统的变化最小。此外，必须对数据进行清理以删除敏感信息并进行处理以使其能够被广泛的研究人员解释。该项目还将提供分析工具来回答某些常见问题，例如工作负载和故障之间的相关性以及使用一个库相对于另一个库的性能影响，以及直观的图形前端，使人们能够探索数据集并下载相关数据集。数据和相关分析工具的广泛使用将为计算机系统研究人员提供前所未有的能力来进行数据驱动的研究，并为计算基础设施提供商提供 分析驱动的能力来运行更高效可靠的基础设施。