Characterizing the Usage of CI Tools in ML Projects
Characterizing the Usage of CI Tools in ML Projects
复制标题
表征 ML 项目中 CI 工具的使用
DOI：
10.1145/3544902.3546237
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Nagappan, Nachiappan
中科院分区：
文献类型：
--
作者：
Rzig, Dhia Elhaq;Hassan, Foyzul;Bansal, Chetan;Nagappan, Nachiappan
背景持续集成（CI）已被广泛采用，以实现更快的代码更改集成。与此同时，机器学习（ML）正在被软件应用程序用于解决以前无法解决的现实世界场景。ML项目采用的开发过程与传统的软件项目不同，但是它们在开发过程中也需要多次迭代，并且可能从CI中受益。虽然在传统软件中有很多关于持续集成的著作，但没有一个是经验性地探讨持续集成的采用及其在机器学习项目中的相关问题。为了解决这一知识差距，我们对ML和非ML项目之间的CI采用进行了实证分析。方法开发了第一个Travis CI配置分析器TraVanalyzer，用于分析ML项目的CI实践，并开发了一个CI日志分析器来识别ML项目的不同CI问题。结果我们发现Travis CI是ML项目中最受欢迎的CI工具，他们的CI采用落后于非ML项目，但采用CI的ML项目比非ML项目更多地使用它进行构建、测试、代码分析和自动部署。此外，虽然ML项目中的CI与非ML项目中的CI一样可能遇到问题，但它有更多不同的构建破坏原因。ML项目中最常见的CI失败是由于与测试相关的问题，类似于非ML和OSS CI失败。据我们所知，这是第一个分析机器学习项目的CI使用、实践和问题的工作，并通过将其与类似的非机器学习项目进行比较，将其结果置于环境中。它为研究人员和机器学习开发人员提供了发现，以确定机器学习项目中CI的可能改进范围。
BackgroundContinuous Integration (CI) has become widely adopted to enable faster code change integration. Meanwhile, Machine Learning (ML) is being used by software applications for previously unsolvable real-world scenarios. ML projects employ development processes different from those of traditional software projects, but they too require multiple iterations in their development, and may benefit from CI.AimsWhile there are many works covering CI within traditional software, none of them empirically explored the adoption of CI and its associated issues within ML projects. To address this knowledge gap, we performed an empirical analysis comparing CI adoption between ML and Non-ML projects.MethodWe developed TraVanalyzer, the first Travis CI configuration analyzer, to analyze the CI practices of ML projects, and developed a CI log analyzer to identify the different CI problems of ML projects.ResultsWe found that Travis CI is the most popular CI tool for ML projects, and that their CI adoption lags behind that of Non-ML projects, but that ML projects which adopted CI, used it for building, testing, code analysis, and automatic deployment more than Non-ML projects. Furthermore, while CI in ML projects is as likely to experience problems as CI in Non-ML projects, it has more varied reasons for build-breakage. The most frequent CI failures of ML projects are due to testing-related problems, similar to Non-ML and OSS CI failures.ConclusionTo the best of our knowledge, this is the first work that has analyzed ML projects’ CI usage, practices, and issues, and contextualized its results by comparing them with similar Non-ML projects. It provides findings for researchers and ML developers to identify possible improvement scopes for CI in ML projects.
登录
查看更多内容
Travis CI 3500 万个工作岗位分析
DOI：
10.1109/icsme.2019.00044
发表时间：
2019
期刊：
2019 IEEE International Conference on Software Maintenance and Evolution (ICSME)
影响因子：
--
作者：
Monperrus Martin;Thomas Durieux;Rui Abreu;Tegawendé F. Bissyandé;Luis Cruz
通讯作者：
Luis Cruz
持续集成中的工作实践和挑战：对 Travis CI 用户的调查
DOI：
10.1002/spe.2637
发表时间：
2018
期刊：
Software: Practice and Experience
影响因子：
--
作者：
G. Pinto;F. C. Filho;R. Bonifácio;Marcel Rebouças
通讯作者：
Marcel Rebouças
DOI：
--
发表时间：
2003-12
期刊：
Journal of Computing Sciences in Colleges
影响因子：
--
作者：
M. Olan
通讯作者：
M. Olan
DOI：
10.1145/356674.356676
发表时间：
1976-09
期刊：
ACM Comput. Surv.
影响因子：
--
作者：
L. Fosdick;L. Osterweil
通讯作者：
L. Fosdick;L. Osterweil
AI 的 DevOps – 开发支持 AI 的应用程序面临的挑战
DOI：
10.23919/softcom50211.2020.9238323
发表时间：
2020
期刊：
2020 International Conference on Software, Telecommunications and Computer Networks (SoftCOM)
影响因子：
--
作者：
Lucy Ellen Lwakatare;I. Crnkovic;J. Bosch
通讯作者：
J. Bosch