CRII: SHF: Assessing and Profiling Continuous Integration for Machine Learning Applications

CRII：SHF：评估和分析机器学习应用程序的持续集成

• 批准号: 2152819
• 负责人: Foyzul Hassan
• 金额: $ 17.43万
• 依托单位: Regents of the University of Michigan - Dearborn
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-05-01 至 2025-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2152819&HistoricalAwards=false
• 关键词: CRII; SHF; Assessing; Profiling; Continuous

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Continuous integration (CI) is a widely adopted software development practice for faster code change integration and maintenance of software quality attributes. At the same time, machine learning (ML), including deep learning (DL), is quickly gaining popularity for solving complex problems. Like typical software, ML applications also require many iterations to improve software quality. However, iterative ML application-development processes face higher-level difficulties in adopting CI in three aspects. First, developers lack systematic understanding for managing ML data, models and code in the CI workflow. For ML-based systems, the process to define the CI workflow is currently much more experimental in nature. Second, existing CI systems are lacking in the handling of ML-centric challenges such as defining evaluation conditions of ML models, formulating complicated build steps, long build and integration time, etc. For ML applications, the build process is much more complicated due to the complex dependency of data, model, code, etc. Third, data scientists lack the technical support to adopt and maintain CI configurations due to complex interconnections among data, model, code, etc. and the changing nature of the ML applications. For data scientists with limited or no knowledge of CI, it becomes increasingly difficult to adopt CI for ML applications, and even if adopted it requires too much manual effort. The project will make progress in acquiring knowledge on the feasibility and effectiveness of the current adoption of CI for ML applications. This will assist in understanding the workflow of CI for ML applications and identifying improvement scopes. Moreover, the project will develop a novel CI profiling framework to generate a dependency graph among heterogeneous artifacts (e.g., data, model, code, etc.) of ML applications. The knowledge and framework will serve as the basis for future research on automatic generation and maintenance of ML CI configuration, mining software repositories, build optimization and monitoring systems for ML CI.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项的全部或部分资金来自《2021年美国救援计划法案》(公共法律117-2)。持续集成(CI)是一种被广泛采用的软件开发实践，用于更快地更改代码、集成和维护软件质量属性。与此同时，机器学习(ML)，包括深度学习(DL)，在解决复杂问题方面正在迅速普及。与典型的软件一样，ML应用程序也需要多次迭代来提高软件质量。然而，迭代的ML应用程序开发过程在三个方面面临采用CI的更高级别的困难。首先，开发人员对在CI工作流程中管理ML数据、模型和代码缺乏系统的了解。对于基于ML的系统，定义CI工作流的过程目前更多的是试验性的。第二，现有的CI系统缺乏处理以ML为中心的挑战，如定义ML模型的评估条件，制定复杂的构建步骤，构建和集成时间较长等。对于ML应用，由于数据、模型、代码等的复杂依赖，构建过程更加复杂。第三，由于数据、模型、代码等之间的复杂互联和ML应用性质的变化，数据科学家缺乏采用和维护CI配置的技术支持。对于对CI了解有限或不了解CI的数据科学家来说，为ML应用程序采用CI变得越来越困难，即使采用，也需要太多的手动工作。该项目将在获取关于目前采用信息技术应用于多语言应用的可行性和有效性方面取得进展。这将有助于理解用于ML应用程序的CI的工作流程，并确定改进范围。此外，该项目将开发一个新的CI概要分析框架，以在不同的构件(例如，数据、模型、代码等)之间生成依赖图。ML应用程序的一部分。该知识和框架将作为未来自动生成和维护ML CI配置、挖掘软件库、为ML CI构建优化和监控系统的研究的基础。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

An Empirical Study of High Performance Computing (HPC) Performance Bugs

• DOI: 10.1109/msr59073.2023.00037
• 发表时间: 2023-05
• 期刊: 2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR)
• 作者: M. Azad;Nafees Iqbal;Foyzul Hassan;Probir Roy

UniLoc: Unified Fault Localization of Continuous Integration Failures

• DOI: 10.1145/3593799
• 发表时间: 2023-05
• 期刊: ACM Transactions on Software Engineering and Methodology
• 影响因子: 4.4
• 作者: Foyzul Hassan;Na Meng;Xiaoyin Wang

Virtual Reality (VR) Automated Testing in the Wild: A Case Study on Unity-Based VR Applications

虚拟现实 (VR) 野外自动测试：基于 Unity 的 VR 应用案例研究

• DOI: 10.1145/3597926.3598134
• 发表时间: 2023
• 期刊: ACM
• 作者: Rzig, Dhia Elhaq;Iqbal, Nafees;Attisano, Isabella;Qin, Xue;Hassan, Foyzul
• 通讯作者: Hassan, Foyzul

An empirical study on ML DevOps adoption trends, efforts, and benefits analysis

• DOI: 10.1016/j.infsof.2022.107037
• 发表时间: 2022-08
• 期刊: Inf. Softw. Technol.
• 作者: D. Rzig;Foyzul Hassan;Marouane Kessentini

Characterizing the Usage of CI Tools in ML Projects

表征 ML 项目中 CI 工具的使用

• DOI: 10.1145/3544902.3546237
• 发表时间: 2022
• 期刊: ESEM '22: Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement
• 作者: Rzig, Dhia Elhaq;Hassan, Foyzul;Bansal, Chetan;Nagappan, Nachiappan
• 通讯作者: Nagappan, Nachiappan