CSR: Medium:Collaborative Research:Holistic, Cross-Site, Hybrid System Anomaly Debugging for Large Scale Hosting Infrastructures

通过允许用户以经济高效的方式按需租用资源，大型共享托管基础设施(如多租户云计算系统)已变得越来越受欢迎。由于多租户可能共享计算资源，托管基础设施是一个复杂的系统，容易出现各种系统异常。尽管软件开发人员经常执行严格的离线测试，但许多细微的错误只是在大规模生产运行期间才显现出来。许多异常现象，例如系统没有崩溃，但未能按预期运行的情况，很难使用现有技术重现和诊断。现有的系统异常诊断工作大致可以分为两类：1)黑盒方案，不需要源代码，适合在线生产现场诊断；2)白盒方案，需要源代码和昂贵的代码插装，适合开发现场，离线诊断。虽然白盒方案提供了细粒度的诊断，但大型生产托管基础设施由于其高开销和侵入性的系统记录方法而不愿采用白盒方案，该项目的总体目标是探索一种创新的跨站点系统异常调试方法，将生产现场黑盒诊断和开发站点白盒调试智能地集成到一个更强大的托管基础设施调试框架中。该项目将开发开发现场离线白盒调试技术，以生产现场故障推断结果为指导，找到准确的异常原因。该项目将专注于诊断非崩溃系统异常(例如，性能下降、服务中断、软件挂起、意外停止)，这些异常在现实世界的托管基础设施中很常见，但很难使用现有技术进行调试。该项目中开发的技术将对提高真实世界托管基础设施的健壮性产生重大影响。专业人员将为他们定期教授的研究生和本科生班级开发关于主机基础设施调试的新课程模块。本项目将在本项目开发的研究原型的基础上开发程序设计课件。PIs将利用其榜样的力量和一系列外展活动来招募更多的女学生从事系统研究。指标将通过出版物和技术转让广泛传播其结果和收集到的数据。开发的软件制品和实验数据集将发布供公众使用。

Large-scale shared hosting infrastructures such as multi-tenant cloud computing systems have become increasingly popular by allowing users to lease resources on-demand in a cost-effective way. As multiple tenants may share computing resources, hosting infrastructures are complex systems and prone to various system anomalies. Although software developers often perform rigorous offline testing, many subtle bugs only manifest themselves during large-scale production run. Many anomalies such as those where the system does not crash but fails to behave as expected are hard to reproduce and diagnose using existing techniques. Existing system anomaly diagnosis work can be broadly classified into two categories: 1) the black-box schemes which do not require source code and are suitable for online production-site diagnosis, and 2) the white-box schemes which require source code and expensive code instrumentation and are suitable for development site, offline diagnosis. Although white-box schemes provide fine-grained diagnosis, large-scale production hosting infrastructures are reluctant to adopt them due to their high-overhead and intrusive system recording approaches.The overarching objective of this project is to explore an innovative cross-site system anomaly debugging approach that intelligently integrates production-site black-box diagnosis with development-site white-box debugging into a more powerful hosting infrastructure debugging framework. This project will develop techniques for development-site, offline white-box debugging that takes the production-site fault inference results as guidance to find the exact anomaly causes. The project will focus on diagnosing non-crashing system anomalies (e.g., performance degradation, service outage, software hang, unexpected halt) that are common in real world hosting infrastructures but are difficult to debug using existing techniques. Techniques developed in this project will generate significant impact on improving the robustness of real world hosting infrastructures. The PIs will develop new course modules on the hosting infrastructure debugging for both graduate and undergraduate classes they regularly teaches. This project will develop programming courseware based on the research prototypes developed in this project. The PIs will use their power of role model and a set of outreach activities to recruit more female students to pursue systems research. The PIs will disseminate their results and collected data broadly through publication and technology transfer. Developed software artifacts and experimental datasets will be released for public use.