CSR: Small: Monitoring for Error Detection in Today's High Throughput Applications

CSR：小：今天的错误检测监控？摘要：我们的许多关键基础设施都是由具有实时性要求的分布式系统构成的。在电力系统、空中交通管制、银行和铁路信号系统中提供关键服务的系统停机可能是灾难性的。错误可能来自单个软件组件，多个组件之间的交互，或者这些组件的错误配置。因此，必须构建低延迟检测系统，该系统随后可以触发诊断和恢复阶段，从而使系统对故障具有鲁棒性。一种强大的错误检测方法是有状态方法，在这种方法中，错误检测系统通过聚合多个消息来构建与应用程序相关的状态。然后规则基于状态，因此基于聚合信息而不是瞬时信息。尽管状态检测的优点似乎被广泛接受，但随着应用程序组件数量的增加或数据速率的增加，很难扩展状态检测。这是由于跟踪应用程序状态和基于状态的规则匹配的处理负载增加。在这个项目中，我们通过设计一个专注于高吞吐量分布式应用程序的运行时监控系统来解决这个问题。我们的解决方案基于智能采样、应用程序状态的概率推理以及对重型规则的机会性监控。一个成功的解决方案将允许高带宽分布式应用程序和具有大量消费者的应用程序可靠地运行。我们还将通过普渡大学一个名为EPICS的创新服务学习项目和一门新课程产生更广泛的影响。

CSR: Small: Monitoring for Error Detection in Today?s High Throughput ApplicationsAbstract: Much of our critical infrastructure is formed by distributed systems with real-time requirements. Downtime of a system providing critical services in power systems, air traffic control, banking, and railways signaling could be catastrophic. The errors may come from individual software components, interactions between multiple components, or misconfiguration of these components. It is therefore imperative to build low latency detection systems that can subsequently trigger the diagnosis and recovery phases leading to systems that are robust to failures. A powerful approach for error detection is the stateful approach, in which the error detection system builds up state related to the application by aggregating multiple messages. The rules are then based on the state, thus on aggregated information rather than on instantaneous information. Though the merits of stateful detection seem to be well accepted, it is difficult to scale stateful detection with an increasing number of application components or increasing data rate. This is due to the increased processing load of tracking application state and rule matching based on the state. In this project, we address this issue through designing a runtime monitoring system focused on high throughput distributed applications. Our solution is based on intelligent sampling, probabilistic reasoning on the application state, and opportunistic monitoring of the heavy-duty rules. A successful solution will allow reliable operation of high bandwidth distributed applications and those with a large number of consumers. We will also achieve broader impact through an innovative service learning program at Purdue called EPICS and a new course.