CSR: Medium:Collaborative Research:Holistic, Cross-Site, Hybrid System Anomaly Debugging for Large Scale Hosting Infrastructures

大型共享托管基础设施（如多租户云计算系统）允许用户以经济有效的方式按需租赁资源，因此变得越来越流行。由于多个租户可能共享计算资源，因此托管基础设施是一个复杂的系统，容易出现各种系统异常。尽管软件开发人员经常执行严格的离线测试，但许多细微的错误只有在大规模生产运行时才会显现出来。许多异常，比如那些系统没有崩溃但没有按照预期运行的异常，很难用现有的技术来重现和诊断。现有的系统异常诊断工作大致可以分为两类：1)不需要源代码的黑盒方案，适用于生产现场在线诊断；2)需要源代码和昂贵的代码检测的白盒方案，适用于开发现场离线诊断。尽管白盒方案提供了细粒度的诊断，但由于其高开销和侵入性的系统记录方法，大型生产托管基础设施不愿意采用它们。本项目的总体目标是探索一种创新的跨站点系统异常调试方法，该方法将生产站点的黑盒诊断与开发站点的白盒调试智能地集成到一个更强大的托管基础设施调试框架中。本项目将开发以生产现场故障推断结果为指导，寻找准确异常原因的开发现场、离线白盒调试技术。该项目将专注于诊断非崩溃系统异常（例如，性能下降、服务中断、软件挂起、意外停止），这些异常在现实世界的托管基础设施中很常见，但很难使用现有技术进行调试。本项目开发的技术将对改善现实世界托管基础设施的健壮性产生重大影响。pi将为他们定期教授的研究生和本科生开发新的托管基础设施调试课程模块。本项目将基于本项目开发的研究原型开发编程课件。pi将利用他们的榜样力量和一系列外展活动来招募更多的女学生从事系统研究。pi将通过出版物和技术转让广泛传播其结果和收集的数据。开发的软件工件和实验数据集将发布供公众使用。

Large-scale shared hosting infrastructures such as multi-tenant cloud computing systems have become increasingly popular by allowing users to lease resources on-demand in a cost-effective way. As multiple tenants may share computing resources, hosting infrastructures are complex systems and prone to various system anomalies. Although software developers often perform rigorous offline testing, many subtle bugs only manifest themselves during large-scale production run. Many anomalies such as those where the system does not crash but fails to behave as expected are hard to reproduce and diagnose using existing techniques. Existing system anomaly diagnosis work can be broadly classified into two categories: 1) the black-box schemes which do not require source code and are suitable for online production-site diagnosis, and 2) the white-box schemes which require source code and expensive code instrumentation and are suitable for development site, offline diagnosis. Although white-box schemes provide fine-grained diagnosis, large-scale production hosting infrastructures are reluctant to adopt them due to their high-overhead and intrusive system recording approaches.The overarching objective of this project is to explore an innovative cross-site system anomaly debugging approach that intelligently integrates production-site black-box diagnosis with development-site white-box debugging into a more powerful hosting infrastructure debugging framework. This project will develop techniques for development-site, offline white-box debugging that takes the production-site fault inference results as guidance to find the exact anomaly causes. The project will focus on diagnosing non-crashing system anomalies (e.g., performance degradation, service outage, software hang, unexpected halt) that are common in real world hosting infrastructures but are difficult to debug using existing techniques. Techniques developed in this project will generate significant impact on improving the robustness of real world hosting infrastructures. The PIs will develop new course modules on the hosting infrastructure debugging for both graduate and undergraduate classes they regularly teaches. This project will develop programming courseware based on the research prototypes developed in this project. The PIs will use their power of role model and a set of outreach activities to recruit more female students to pursue systems research. The PIs will disseminate their results and collected data broadly through publication and technology transfer. Developed software artifacts and experimental datasets will be released for public use.