III:Small:Integrated Problem Diagnosis and Repair in Databases and Storage Area Networks

数据库通常用作包含Web服务器、应用程序服务器和网络附加存储服务器的大型系统中的子系统。这种复杂的系统总是经历某种形式的变化，例如，应用服务器中的Java模块的更新，数据库中的统计数据更新，或者存储卷中的RAID重建。不同子系统中的这种变化可能导致整体性能下降，其原因很难诊断。由于企业环境对每个子系统都有独立的管理团队和工具，因此诊断任务更加艰巨。该项目正在开发一个名为DIADS的集成工具，该工具可以自动执行复杂的管理任务，如问题诊断、假设分析、编排灾难恢复，以及当数据库在更大的系统中用作子系统时的在线调优。DIADS包含两项技术创新。问题诊断包括使用从系统收集的历史和当前监控数据在各个时间点重构系统行为。然而，由于需要保持较低的监控开销，生产系统中可用的监控数据的数量和质量受到限制。DIADS使用一种称为标注规划图（Annotated Plan Graph）的抽象来表示和推理更大系统上下文中的数据库行为。标注的PlanGraphs是由轻量级的监控数据生成的。DIADS的另一个创新是一套管理任务的工作流程，它将机器学习技术与系统专家的领域知识相结合。例如，对于问题诊断，工作流程的机器学习部分提供了核心技术来处理大量嘈杂的监控数据流，而领域知识部分则充当了检查和平衡的角色，以指导诊断朝着正确的方向发展。这种独特的设计使DIADS即使在存在多个并发问题以及生产环境中普遍存在的嘈杂监控数据的情况下也能有效地工作。DIADS的原型是用于研究和教育目的，在数据中心设置PostgreSQL数据库和企业级存储区域网络。欲了解更多信息，请参阅该项目的网页http://www.cs.duke.edu/~shivnath/diads.html

Databases are typically used as a subsystem in a larger system thatcontains Web servers, application servers, and network-attachedstorage servers. Such complex systems experience some form of changeall the time, e.g., an update to a Java module in the applicationserver, a statistics update in the database, or a RAID rebuild in astorage volume. Such changes in different subsystems can cause anoverall performance degradation whose cause is hard to diagnose. Thediagnosis task is all the more daunting because enterpriseenvironments have isolated administration teams and tools for eachsubsystem.This project is developing an integrated tool called DIADS thatautomates complex administrative tasks like problem diagnosis, what-ifanalysis, orchestrating disaster recovery, and online tuning when adatabase is used as a subsystem in a larger system. DIADS contains twotechnical innovations. Problem diagnosis involves reconstructingsystem behavior at various points of time using historic and currentmonitoring data collected from the system. However, the amount andquality of monitoring data available from production systems isconstrained by the need to keep monitoring overhead low. DIADS uses anabstraction called Annotated Plan Graph to represent and reason aboutdatabase behavior in the context of a larger system. Annotated PlanGraphs are generated from light-weight monitoring data.The other innovation in DIADS is a suite of workflows foradministrative tasks that combine machine-learning techniques withdomain knowledge from system experts. For example, for problemdiagnosis, the machine-learning part of the workflow provides coretechniques to handle large and noisy streams of monitoring data, whilethe domain-knowledge part acts as checks-and-balances to guide thediagnosis in the right direction. This unique design enables DIADS tofunction effectively even in the presence of multiple concurrentproblems as well as noisy monitoring data prevalent in productionenvironments. DIADS is being prototyped for research and educationalpurposes in a datacenter setting with PostgreSQL databases and anenterprise-level storage area network. For further information see the project web page at http://www.cs.duke.edu/~shivnath/diads.html