CSR: Medium: Augmenting Logs with Static Analysis and Symbolic Execution

Facebook、亚马逊、Gmail、iTunes或Office 360等互联网服务是现代经济的支柱，包括由大中小型企业提供的数百万个消费者应用程序和商业服务。所有这些服务本质上都被组织为复杂的分布式软件系统。当这样的系统遇到故障(服务中断或意外的性能问题)时，故障的潜在成本可能很快达到数百万美元的收入损失或生产力损失。快速诊断问题的根本原因和尽快恢复服务都是至关重要的。解决此问题的基本方法是在系统组件执行和交互时频繁记录(或记录)信息，然后在尝试诊断根本原因失败后分析这些日志。虽然这些分析使用越来越复杂的统计和基于机器学习的技术，但它们受到日志中收集的信息的限制：目前最先进的方法记录与重大事件有关的有限信息，以便在正常操作期间不会显著降低服务速度。特别是，这些系统不记录组件内的详细内部信息或组件之间的详细网络流量，这对更快地诊断故障非常有用。该项目开发了新的技术，通过利用程序分析和自动化测试的最新进展，使重建这些缺失的信息成为可能。此外，该项目开发了一种新的策略，将重建任务分解为几个小步骤，使分析能够扩展到非常大的分布式服务。该项目还为个别组件开发了新的、更复杂的分析技术，这些技术超出了当前最先进的水平，根据这项工作中面临的具体问题进行了调整。该项目邀请行业合作伙伴评估其系统上的技术，并以开放源代码的形式向研究界和行业合作伙伴提供所产生的软件解决方案。总而言之，这些技术有可能为广泛的互联网服务节省大量成本。

Internet services such as Facebook, Amazon, Gmail, iTunes, or Office 360 are a backbone of the modern economy, and include millions of consumer applications and business services provided by large, medium-sized and small businesses. All these services are essentially organized as complex distributed software systems. When such a system encounters a failure (a service outage or an unexpected performance problem), the potential costs of the failure can quickly reach millions of dollars in lost revenue or lost productivity. Diagnosing the root causes of the problem quickly and restoring the service as soon as possible are both critical.The basic approach to this problem is to record (or "log") information frequently as system components execute and interact, and then to analyze these logs after a failure in attempting to diagnose the root causes. While these analyses use increasingly sophisticated statistical and machine-learning based techniques, they are limited by the information gathered in the logs: the state-of-the-art approaches today record limited information pertaining to major events so that they do not slow down the services significantly during normal operation. In particular, these systems do not record detailed internal information within components or detailed network traffic between components that can be enormously useful in diagnosing failures more quickly.This project develops new techniques that make it possible to reconstruct this missing information by leveraging recent advances in program analysis and automated testing. Moreover, the project develops a new strategy that breaks down the reconstruction task into small steps that will allow the analysis to scale to very large distributed services. The project also develops new, more sophisticated analysis techniques for individual components that go beyond the current state-of-the-art, by tailoring them to the specific problems faced in this work. The project engages industrial partners to evaluate the techniques on their systems, and makes the resulting software solutions available in open source form to the research community as well as the industrial partners. Together, these techniques have the potential to result in large cost-savings for a wide range of Internet services.