SHF: Small: Failure Events Modeling and Analysis for Proactive Management in Highly Dependable Systems

SHF：小型：高度可靠系统中主动管理的故障事件建模和分析

• 批准号: 1016966
• 负责人: Chengzhong Xu
• 金额: $ 46.78万
• 依托单位: Wayne State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-09-01 至 2015-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1016966&HistoricalAwards=false
• 关键词: SHF; Small; Failure; Events; Modeling

In large-scale computer systems, component failures are no longer rare events. As the scale of the systems continues to increase, their reliability and service availability become an increasingly critical concern. Recent IT expenditure analyses also show that the worldwide spending in server management and administration has surpassed the cost of new server acquisition. Conventional reactive trouble-shooting measures and conservative check-pointing approaches are often counter-productive or may cause a long time service disruption. The goal of this FEMA project is to develop modeling and analytical methodologies and tools to characterize the systems failure dynamics for proactive failure management in highly dependable systems.This FEMA project is carried out in three aspects. First is the development of an aggregated spherical covariance model that characterizes the failure dynamics quantitatively. The model centers on a failure signature concept that correlates a group of OS-level performance parameters and operation-level job allocation information to different types of fault events in both space and time domains. Second is an innovative application of statistical learning methods for failure prediction. Different failures types in different system scopes have different failure dynamics and different amount of history data for training; different prediction metrics pose different requirements for prediction granularity. Various supervised, unsupervised, and reinforcement learning algorithms find their applications in different scenarios. Third is the development of system reliability traces for offline evaluation and a methodology for online prediction in production systems. The trace not only contains a log of failure events, but also their corresponding operational contexts that are necessary for attaining high prediction accuracy.

在大型计算机系统中，组件故障不再是罕见事件。随着系统规模的不断增加，其可靠性和服务可用性成为越来越重要的问题。最近的 IT 支出分析还表明，全球在服务器管理方面的支出已经超过了购买新服务器的成本。传统的反应式故障排除措施和保守的检查点方法通常会适得其反，或者可能导致长时间的服务中断。该 FEMA 项目的目标是开发建模和分析方法和工具来表征系统故障动态，以便在高度可靠的系统中进行主动故障管理。该 FEMA 项目分三个方面进行。首先是开发聚合球形协方差模型，定量表征失效动态。该模型以故障签名概念为中心，将一组操作系统级性能参数和操作级作业分配信息与空间和时间域中不同类型的故障事件相关联。其次是统计学习方法在故障预测方面的创新应用。不同系统范围内的不同故障类型，其故障动态不同，用于训练的历史数据量也不同；不同的预测指标对预测粒度提出了不同的要求。各种监督、无监督和强化学习算法在不同的场景中都有各自的应用。第三是开发用于离线评估的系统可靠性跟踪和生产系统在线预测的方法。跟踪不仅包含故障事件的日志，还包含获得高预测精度所必需的相应操作上下文。