CSR: Medium:Collaborative Research:Holistic, Cross-Site, Hybrid System Anomaly Debugging for Large Scale Hosting Infrastructures

诸如多租户云计算系统之类的大规模共享托管基础设施通过允许用户以具有成本效益的方式按需租赁资源而变得越来越流行。由于多个租户可以共享计算资源，托管基础设施是复杂的系统，并且容易出现各种系统异常。尽管软件开发人员经常执行严格的离线测试，但许多细微的错误只在大规模生产运行期间才会显现出来。许多异常，如系统没有崩溃，但未能按照预期的行为是很难重现和诊断使用现有的技术。现有的系统异常诊断工作大致可以分为两类：1）不需要源代码、适合生产现场在线诊断的黑盒方案; 2）需要源代码和昂贵代码插装、适合开发现场离线诊断的白盒方案。虽然白盒方案提供细粒度的诊断，大规模的生产托管基础设施不愿意采用它们，因为它们的开销很高，而且会侵入系统记录方法。本项目的总体目标是探索一种创新的跨站点系统异常调试方法，该方法智能地集成了生产站点的黑盒诊断和开发站点的白色诊断-框调试到一个更强大的托管基础设施调试框架。本项目将开发开发现场离线白盒调试技术，以生产现场故障推断结果为指导，找到确切的异常原因。该项目将侧重于诊断非崩溃系统异常（例如，性能降级、服务中断、软件挂起、意外停机），这些在真实的世界主机基础结构中是常见的，但是难以使用现有技术进行调试。本项目开发的技术将对提高真实的世界托管基础设施的健壮性产生重大影响。PI将为他们定期教授的研究生和本科生课程开发关于托管基础设施调试的新课程模块。本计画将以本计画所开发之研究原型为基础，开发程式设计课件。研究员将利用他们的榜样力量和一系列外展活动，招募更多女学生从事系统研究。参与者将通过出版物和技术转让广泛传播其结果和收集的数据。开发的软件工件和实验数据集将被发布供公众使用。

Large-scale shared hosting infrastructures such as multi-tenant cloud computing systems have become increasingly popular by allowing users to lease resources on-demand in a cost-effective way. As multiple tenants may share computing resources, hosting infrastructures are complex systems and prone to various system anomalies. Although software developers often perform rigorous offline testing, many subtle bugs only manifest themselves during large-scale production run. Many anomalies such as those where the system does not crash but fails to behave as expected are hard to reproduce and diagnose using existing techniques. Existing system anomaly diagnosis work can be broadly classified into two categories: 1) the black-box schemes which do not require source code and are suitable for online production-site diagnosis, and 2) the white-box schemes which require source code and expensive code instrumentation and are suitable for development site, offline diagnosis. Although white-box schemes provide fine-grained diagnosis, large-scale production hosting infrastructures are reluctant to adopt them due to their high-overhead and intrusive system recording approaches.The overarching objective of this project is to explore an innovative cross-site system anomaly debugging approach that intelligently integrates production-site black-box diagnosis with development-site white-box debugging into a more powerful hosting infrastructure debugging framework. This project will develop techniques for development-site, offline white-box debugging that takes the production-site fault inference results as guidance to find the exact anomaly causes. The project will focus on diagnosing non-crashing system anomalies (e.g., performance degradation, service outage, software hang, unexpected halt) that are common in real world hosting infrastructures but are difficult to debug using existing techniques. Techniques developed in this project will generate significant impact on improving the robustness of real world hosting infrastructures. The PIs will develop new course modules on the hosting infrastructure debugging for both graduate and undergraduate classes they regularly teaches. This project will develop programming courseware based on the research prototypes developed in this project. The PIs will use their power of role model and a set of outreach activities to recruit more female students to pursue systems research. The PIs will disseminate their results and collected data broadly through publication and technology transfer. Developed software artifacts and experimental datasets will be released for public use.