Adoption and Effects of Software Engineering Best Practices in Machine Learning
Adoption and Effects of Software Engineering Best Practices in Machine Learning
复制标题
机器学习中软件工程最佳实践的采用和效果
DOI：
10.1145/3382494.3410681
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Joost Visser
中科院分区：
文献类型：
--
作者：
A. Serban;K. Blom;H. Hoos;Joost Visser
背景。人们越来越依赖具有机器学习 (ML) 组件的应用程序，这就需要成熟的工程技术来确保这些应用程序以稳健且面向未来的方式构建。目的。我们的目标是根据经验确定团队如何使用机器学习组件开发、部署和维护软件的最新技术水平。方法。我们挖掘了学术文献和灰色文献，并确定了 29 个 ML 应用的工程最佳实践。我们对 313 名从业者进行了一项调查，以确定这些做法的采用程度并验证其感知效果。利用调查回复，我们量化了实践采用情况，并根据人口特征（例如地理位置或团队规模）进行了区分。我们还使用各种统计模型测试了相关性并研究了实践及其感知效果之间的线性和非线性关系。结果。例如，我们的研究结果表明，较大的团队往往会采用更多的实践，而传统的软件工程实践的采用率往往低于 ML 特定实践。此外，统计模型可以根据特定实践集的采用程度准确预测感知效果，例如敏捷性、软件质量和可追溯性。将实践采用率与实践重要性相结合（如统计模型所示），我们确定了重要但采用率较低的实践，以及广泛采用但对我们研究的效果不太重要的实践。结论。总的来说，我们的调查和对收到的回复的分析为机器学习团队对实践采用的评估和逐步改进提供了定量基础。
Background. The increasing reliance on applications with machine learning (ML) components calls for mature engineering techniques that ensure these are built in a robust and future-proof manner. Aim. We aim to empirically determine the state of the art in how teams develop, deploy and maintain software with ML components. Method. We mined both academic and grey literature and identified 29 engineering best practices for ML applications. We conducted a survey among 313 practitioners to determine the degree of adoption for these practices and to validate their perceived effects. Using the survey responses, we quantified practice adoption, differentiated along demographic characteristics, such as geography or team size. We also tested correlations and investigated linear and non-linear relationships between practices and their perceived effect using various statistical models. Results. Our findings indicate, for example, that larger teams tend to adopt more practices, and that traditional software engineering practices tend to have lower adoption than ML specific practices. Also, the statistical models can accurately predict perceived effects such as agility, software quality and traceability, from the degree of adoption for specific sets of practices. Combining practice adoption rates with practice importance, as revealed by statistical models, we identify practices that are important but have low adoption, as well as practices that are widely adopted but are less important for the effects we studied. Conclusion. Overall, our survey and the analysis of responses received provide a quantitative basis for assessment and step-wise improvement of practice adoption by ML teams.
影响因子：
23.8
作者：
Lundberg, Scott M.;Erion, Gabriel;Lee, Su-In
通讯作者：
Lee, Su-In