Improving the Software Logging Practices to Support the Decision-Making Process of DevOps Engineers

改进软件日志记录实践以支持 DevOps 工程师的决策过程

• 批准号: RGPIN-2020-06122
• 负责人: Jiang, ZhenMing(Jack)
• 金额: $ 2.55万
• 依托单位: York University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=752429
• 关键词: Improving; Software; Logging; Practices; Support

DevOps is currently a popular development methodology that intends to provide rapid feedback between software development and IT operations. Compared to traditional software development processes, DevOps aims to provide rapid feedback between software development (a.k.a., the Devs) and field operations (a.k.a., the Ops) so that new features and bug fixes can be released faster to customers. To ensure the quality and the health of the deployed systems, software logging plays a central role. Logs are ubiquitous. They exist in all types of software systems ranging from embedded devices to ultra-large-scale cloud platforms. According to Gartner, software logging is a 1.5 billion market, which is growing at an annual rate of 10%. Unfortunately, existing logging practices are ad-hoc and rely mainly on DevOps engineers' experience and gut feelings. On the development side, as logging code tangles with feature code, it is very challenging to maintain and update logging code along with feature code for constantly evolving systems. On the operation side, many systems also deploy additional monitoring infrastructures (e.g., Application Performance Monitoring and distributed tracing) on top of software logging, existing log analysis techniques seldom leverage the rich information that is contained in these telemetry data. Furthermore, there are limited mechanisms for knowledge sharing between developers and operators, although they both use logs extensively. In order to achieve my long-term research vision, which is to support data-driven decision making throughout the software development and operations lifecycles, my short-term research objective over the next five-years is to support the decision making process of DevOps engineers by systematically improving the software logging practices. The following three research topics are proposed towards this objective: (1) analytics-driven logging code design, which derives the best practices from existing systems to assist developers in producing high quality logging code, (2) deployment-aware log analysis, which leverages telemetry data to assist operators in root cause isolation, and (3) use-case-oriented sharing of logging knowledge, which integrates development and operation information to facilitate collaboration and interaction between the Devs and the Ops. This research will train twelve HQPs (4 PhD, 3 MSc, and 5 undergraduate students) in the following two highly sought-after professions in the Information and Communication Technology (ICT) sector: (1) DevOps Engineers, and (2) Data Scientists.

DevOps目前是一种流行的开发方法，旨在提供软件开发和IT操作之间的快速反馈。与传统的软件开发流程相比，DevOps旨在提供软件开发（即Devs）和现场操作（即Ops）之间的快速反馈，以便可以更快地向客户发布新功能和错误修复。为了确保已部署系统的质量和运行状况，软件日志记录起着核心作用。原木无处不在。它们存在于从嵌入式设备到超大规模云平台的所有类型的软件系统中。根据Gartner的数据，软件日志是一个15亿美元的市场，并以每年10%的速度增长。不幸的是，现有的日志记录实践是临时的，主要依赖于DevOps工程师的经验和直觉。在开发方面，由于日志代码与功能代码纠缠在一起，对于不断发展的系统来说，维护和更新日志代码以及功能代码是非常具有挑战性的。在操作方面，许多系统还在软件日志记录之上部署了额外的监视基础设施（例如，应用程序性能监视和分布式跟踪），现有的日志分析技术很少利用这些遥测数据中包含的丰富信息。此外，尽管开发人员和操作人员都广泛使用日志，但他们之间的知识共享机制有限。为了实现我的长期研究愿景，即在整个软件开发和操作生命周期中支持数据驱动的决策制定，我在未来五年的短期研究目标是通过系统地改进软件日志实践来支持DevOps工程师的决策制定过程。为此，提出以下三个研究课题：(1)分析驱动的日志代码设计，从现有系统中获得最佳实践，帮助开发人员生成高质量的日志代码；(2)部署感知日志分析，利用遥测数据帮助操作人员隔离根本原因；(3)面向用例的日志知识共享，集成开发和操作信息，促进开发人员和运维人员之间的协作和互动。这项研究将培养12名hqp（4名博士，3名硕士和5名本科生），从事以下两个在信息和通信技术（ICT）领域非常受欢迎的职业：(1)开发运维工程师，(2)数据科学家。