Improving the Software Logging Practices to Support the Decision-Making Process of DevOps Engineers

改进软件日志记录实践以支持 DevOps 工程师的决策过程

• 批准号: RGPIN-2020-06122
• 负责人: Jiang, ZhenMing(Jack)
• 金额: $ 2.55万
• 依托单位: York University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=717726
• 关键词: Improving; Software; Logging; Practices; Support

DevOps is currently a popular development methodology that intends to provide rapid feedback between software development and IT operations. Compared to traditional software development processes, DevOps aims to provide rapid feedback between software development (a.k.a., the Devs) and field operations (a.k.a., the Ops) so that new features and bug fixes can be released faster to customers. To ensure the quality and the health of the deployed systems, software logging plays a central role. Logs are ubiquitous. They exist in all types of software systems ranging from embedded devices to ultra-large-scale cloud platforms. According to Gartner, software logging is a 1.5 billion market, which is growing at an annual rate of 10%. Unfortunately, existing logging practices are ad-hoc and rely mainly on DevOps engineers' experience and gut feelings. On the development side, as logging code tangles with feature code, it is very challenging to maintain and update logging code along with feature code for constantly evolving systems. On the operation side, many systems also deploy additional monitoring infrastructures (e.g., Application Performance Monitoring and distributed tracing) on top of software logging, existing log analysis techniques seldom leverage the rich information that is contained in these telemetry data. Furthermore, there are limited mechanisms for knowledge sharing between developers and operators, although they both use logs extensively. In order to achieve my long-term research vision, which is to support data-driven decision making throughout the software development and operations lifecycles, my short-term research objective over the next five-years is to support the decision making process of DevOps engineers by systematically improving the software logging practices. The following three research topics are proposed towards this objective: (1) analytics-driven logging code design, which derives the best practices from existing systems to assist developers in producing high quality logging code, (2) deployment-aware log analysis, which leverages telemetry data to assist operators in root cause isolation, and (3) use-case-oriented sharing of logging knowledge, which integrates development and operation information to facilitate collaboration and interaction between the Devs and the Ops. This research will train twelve HQPs (4 PhD, 3 MSc, and 5 undergraduate students) in the following two highly sought-after professions in the Information and Communication Technology (ICT) sector: (1) DevOps Engineers, and (2) Data Scientists.

DevOps是目前流行的开发方法，旨在提供软件开发和IT运营之间的快速反馈。与传统的软件开发过程相比，DevOps旨在提供软件开发之间的快速反馈（即，Devs）和现场操作（a.k.a.，Ops），以便可以更快地向客户发布新功能和错误修复。为了确保所部署系统的质量和健康，软件日志记录起着核心作用。 它们无处不在。它们存在于从嵌入式设备到超大规模云平台的所有类型的软件系统中。根据Gartner的数据，软件日志记录是一个15亿美元的市场，每年以10%的速度增长。不幸的是，现有的日志记录实践是临时的，主要依赖于DevOps工程师的经验和直觉。在开发方面，由于日志代码与功能代码纠缠在一起，因此对于不断发展的系统，维护和更新日志代码沿着功能代码是非常具有挑战性的。在操作侧，许多系统还部署额外的监视基础设施（例如，应用性能监视和分布式跟踪），现有的日志分析技术很少利用这些遥测数据中包含的丰富信息。此外，虽然开发人员和运营商都广泛使用日志，但他们之间的知识共享机制有限。 为了实现我的长期研究愿景，即在整个软件开发和运营生命周期中支持数据驱动的决策，我在未来五年的短期研究目标是通过系统地改进软件日志实践来支持DevOps工程师的决策过程。为实现这一目标，提出了以下三个研究课题：（1）分析驱动的日志代码设计，其从现有系统中获得最佳实践以帮助开发人员产生高质量的日志代码，（2）部署感知日志分析，其利用遥测数据来帮助操作员进行根本原因隔离，以及（3）面向用例的日志知识共享，其集成了开发和操作信息以促进开发人员和操作人员之间的协作和交互。这项研究将培养12名HQP（4名博士，3名硕士和5名本科生），从事信息和通信技术（ICT）领域以下两个备受追捧的职业：（1）DevOps工程师和（2）数据科学家。