Improving the Software Logging Practices to Support the Decision-Making Process of DevOps Engineers

点击翻译按钮获取中文摘要

DevOps is currently a popular development methodology that intends to provide rapid feedback between software development and IT operations. Compared to traditional software development processes, DevOps aims to provide rapid feedback between software development (a.k.a., the Devs) and field operations (a.k.a., the Ops) so that new features and bug fixes can be released faster to customers. To ensure the quality and the health of the deployed systems, software logging plays a central role. Logs are ubiquitous. They exist in all types of software systems ranging from embedded devices to ultra-large-scale cloud platforms. According to Gartner, software logging is a 1.5 billion market, which is growing at an annual rate of 10%. Unfortunately, existing logging practices are ad-hoc and rely mainly on DevOps engineers' experience and gut feelings. On the development side, as logging code tangles with feature code, it is very challenging to maintain and update logging code along with feature code for constantly evolving systems. On the operation side, many systems also deploy additional monitoring infrastructures (e.g., Application Performance Monitoring and distributed tracing) on top of software logging, existing log analysis techniques seldom leverage the rich information that is contained in these telemetry data. Furthermore, there are limited mechanisms for knowledge sharing between developers and operators, although they both use logs extensively. In order to achieve my long-term research vision, which is to support data-driven decision making throughout the software development and operations lifecycles, my short-term research objective over the next five-years is to support the decision making process of DevOps engineers by systematically improving the software logging practices. The following three research topics are proposed towards this objective: (1) analytics-driven logging code design, which derives the best practices from existing systems to assist developers in producing high quality logging code, (2) deployment-aware log analysis, which leverages telemetry data to assist operators in root cause isolation, and (3) use-case-oriented sharing of logging knowledge, which integrates development and operation information to facilitate collaboration and interaction between the Devs and the Ops. This research will train twelve HQPs (4 PhD, 3 MSc, and 5 undergraduate students) in the following two highly sought-after professions in the Information and Communication Technology (ICT) sector: (1) DevOps Engineers, and (2) Data Scientists.