Improving the Software Logging Practices to Support the Decision-Making Process of DevOps Engineers

改进软件日志记录实践以支持 DevOps 工程师的决策过程

• 批准号: RGPIN-2020-06122
• 负责人: Jiang, ZhenMing(Jack)
• 金额: $ 2.55万
• 依托单位: York University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=752429
• 关键词: Improving; Software; Logging; Practices; Support

DevOps is currently a popular development methodology that intends to provide rapid feedback between software development and IT operations. Compared to traditional software development processes, DevOps aims to provide rapid feedback between software development (a.k.a., the Devs) and field operations (a.k.a., the Ops) so that new features and bug fixes can be released faster to customers. To ensure the quality and the health of the deployed systems, software logging plays a central role. Logs are ubiquitous. They exist in all types of software systems ranging from embedded devices to ultra-large-scale cloud platforms. According to Gartner, software logging is a 1.5 billion market, which is growing at an annual rate of 10%. Unfortunately, existing logging practices are ad-hoc and rely mainly on DevOps engineers' experience and gut feelings. On the development side, as logging code tangles with feature code, it is very challenging to maintain and update logging code along with feature code for constantly evolving systems. On the operation side, many systems also deploy additional monitoring infrastructures (e.g., Application Performance Monitoring and distributed tracing) on top of software logging, existing log analysis techniques seldom leverage the rich information that is contained in these telemetry data. Furthermore, there are limited mechanisms for knowledge sharing between developers and operators, although they both use logs extensively. In order to achieve my long-term research vision, which is to support data-driven decision making throughout the software development and operations lifecycles, my short-term research objective over the next five-years is to support the decision making process of DevOps engineers by systematically improving the software logging practices. The following three research topics are proposed towards this objective: (1) analytics-driven logging code design, which derives the best practices from existing systems to assist developers in producing high quality logging code, (2) deployment-aware log analysis, which leverages telemetry data to assist operators in root cause isolation, and (3) use-case-oriented sharing of logging knowledge, which integrates development and operation information to facilitate collaboration and interaction between the Devs and the Ops. This research will train twelve HQPs (4 PhD, 3 MSc, and 5 undergraduate students) in the following two highly sought-after professions in the Information and Communication Technology (ICT) sector: (1) DevOps Engineers, and (2) Data Scientists.

DevOps目前是一种流行的开发方法，旨在在软件开发和IT操作之间提供快速反馈。与传统的软件开发过程相比，DevOps旨在在软件开发（又称DEVS）和现场操作（又称OPS）之间提供快速反馈，以便可以更快地向客户发布新功能和错误修复程序。为了确保部署系统的质量和健康，软件记录起着核心作用。 日志无处不在。它们存在于从嵌入式设备到超大规模云平台的所有类型的软件系统中。根据Gartner的说法，软件记录是一个15亿个市场，每年增长10％。不幸的是，现有的记录实践是临时的，主要依靠DevOps工程师的经验和直觉。在开发方面，作为具有功能代码的记录代码缠结，维护和更新记录代码以及用于不断发展的系统的功能代码非常具有挑战性。在操作方面，许多系统还部署了其他监视基础架构（例如，应用程序性能监视和分布式跟踪）在软件日志日期之上，现有的日志分析技术很少利用这些遥测数据中包含的丰富信息。此外，尽管它们都广泛使用日志，但开发人员和运营商之间的知识共享机制有限。 为了实现我的长期研究愿景，即支持整个软件开发和运营生命周期的数据驱动决策，我在接下来的五年内的短期研究目标是通过系统地改善软件记录实践来支持DevOps工程师的决策过程。 The following three research topics are proposed towards this objective: (1) analytics-driven logging code design, which derives the best practices from existing systems to assist developers in producing high quality logging code, (2) deployment-aware log analysis, which leverages telemetry data to assist operators in root cause isolation, and (3) use-case-oriented sharing of logging knowledge, which integrates development and operation information to facilitate collaboration and interaction在开发人员和行动之间。这项研究将在以下两个备受追捧的信息和通信技术（ICT）行业中培训十二个HQP（4博士学位，3名MSC和5名本科生）：（1）DevOps工程师和（2）数据科学家。