SHF:Small: More Modular Deep Learning

SHF:Small：更加模块化的深度学习

• 批准号: 2223812
• 负责人: Hridesh Rajan
• 金额: $ 58万
• 依托单位: Iowa State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2223812&HistoricalAwards=false
• 关键词: SHF; Small; More; Modular; Deep

This project will study a class of machine learning algorithms known as deep learning that has received much attention in academia and industry. Deep learning has a large number of important societal applications, from self-driving cars to question-answering systems such as Siri and Alexa. A deep learning algorithm uses multiple layers of transformation functions to convert inputs to outputs, each layer learning higher-level of abstractions in the data successively. The availability of large datasets has made it feasible to train deep learning models. Since the layers are organized in the form of a network, such models are also referred to as deep neural networks (DNN). While the jury is still out on the impact of deep learning on the overall understanding of software's behavior, a significant uptick in its usage and applications in wide-ranging areas and safety-critical systems, e.g., autonomous driving, aviation system, medical analysis, etc., combine to warrant research on software engineering practices in the presence of deep learning. One challenge is to enable the reuse and replacement of the parts of a DNN that has the potential to make DNN development more reliable. This project will investigate a comprehensive approach to systematically investigate the decomposition of deep neural networks into modules to enable reuse, replacement, and independent evolution of those modules. A module is an independent part of a software system that can be tested, validated, or utilized without a major change to the rest of the system. Allowing the reuse of DNN modules is expected to reduce energy- and data-intensive training efforts to construct DNN models. Allowing replacement is expected to help replace faulty functionality in DNN models without needing costly retraining steps. The preliminary work of the investigator has shown that it is possible to decompose fully connected neural networks and CNN models into modules and conceptualize the notion of modules. The main goals and the intellectual merits of this project are to further expand this decomposition approach along three dimensions: (1) Does the decomposition approach generalize to large Natural Language Processing (NLP) models, where a huge reduction in CO2e emission is expected? (2) What criteria should be used for decomposing a DNN into modules? A better understanding of the decomposition criteria can help inform the design and implementation of DNNs and reduce the impact of changes. (3) While coarse-grained decomposition has worked well for FCNNs and CNNs, does a finer-grained decomposition of DNNs into modules connected using AND-OR-NOT primitives a la structured decomposition has the potential to both enable more reuse (especially for larger DNNs) and provide deeper insights into the behavior of DNNs? The project also incorporates a rigorous evaluation plan using widely studied datasets. The project is expected to broadly impact society by informing the science and practice of deep learning. A serious problem facing the current software development workforce is that deep learning is widely utilized in our software systems, but scientists and practitioners do not yet have a clear handle on critical problems such as explainability of DNN models, DNN reuse, replacement, independent testing, and independent development. There was no apparent need to investigate the notions of modularity as neural network models trained before the deep learning era were mostly small, trained on small datasets, and were mostly used as experimental features. The notion of DNN modules developed by this project, if successful, could help make significant advances on a number of open challenges in this area. DNN modules could enable the reuse of already trained DNN modules in another context. Viewing a DNN as a composition of DNN modules instead of a black box could enhance the explainability of a DNN's behavior. This project, if successful, will thus have a large positive impact on the productivity of these programmers, the understandability and maintainability of the DNN models that they deploy, and the scalability and correctness of software systems that they produce. Other impacts will include: research-based advanced training as well as enhancement in experimental and system-building expertise of future computer scientists, incorporation of research results into courses at Iowa State University as well as facilitating the integration of modularity research-related topics, and increased opportunities for the participation of underrepresented groups in research-based training.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目将研究一种称为深度学习的机器学习算法，该算法在学术界和行业中受到了很多关注。深度学习具有大量重要的社会应用，从自动驾驶汽车到Siri和Alexa等提问系统。深度学习算法使用多个转换函数将输入转换为输出，每个层连续学习了数据中的高级抽象。大型数据集的可用性使训练深度学习模型变得可行。由于这些层是以网络形式组织的，因此此类模型也称为深神经网络（DNN）。虽然陪审团仍在深入学习对软件行为的整体理解的影响方面，但其在广泛领域的使用以及应用程序的应用以及安全至关重要的系统的应用，例如自动驾驶，航空系统，医疗分析等，在深度学习的存在下，合并了对软件工程实践的保证研究。一个挑战是实现有可能使DNN开发更可靠的DNN部分的再利用和替换。该项目将研究一种全面的方法，以系统地研究深神网络将其分解为模块，以实现这些模块的重复使用，替换和独立演变。模块是软件系统的独立部分，可以测试，验证或使用，而无需对系统的其余部分进行重大更改。允许重复使用DNN模块，有望减少构建DNN模型的能源和数据密集型培训工作。允许更换有望帮助替换DNN模型中的错误功能，而无需昂贵的再培训步骤。研究者的初步工作表明，可以将完全连接的神经网络和CNN模型分解为模块，并概念化模块的概念。该项目的主要目标和智力优点是进一步扩展这种分解方法沿三个维度：（1）分解方法是否会推广到大型自然语言处理（NLP）模型，在这种模型中，预计CO2E排放会大大减少？ （2）应使用哪些标准将DNN分解为模块？更好地了解分解标准可以帮助告知DNN的设计和实施并减少变化的影响。 （3）虽然粗粒分解在FCNN和CNNS方面效果很好，但DNN的细粒度分解是否较细化，使用and or-or-or-or-obiritives连接的模块，la结构化分解具有既可以启用更多的重复使用（尤其是对于较大的DNN），并为DNNS的行为提供了更深的洞察力？该项目还使用广泛研究的数据集纳入了严格的评估计划。预计该项目将通过告知深度学习的科学和实践来广泛影响社会。当前软件开发劳动力面临的一个严重问题是，深度学习已被广​​泛使用，但是科学家和从业者尚未清楚地处理关键问题，例如DNN模型的解释性，DNN重用，替换，独立测试和独立开发。显然不需要研究模块化的概念，因为在深度学习时代之前接受过训练的神经网络模型大多是小的，在小型数据集上训练，并且主要用作实验特征。如果成功的话，该项目开发的DNN模块的概念可能有助于在该领域的许多开放挑战上取得重大进步。 DNN模块可以在另一种情况下重新使用已经训练的DNN模块。将DNN视为DNN模块而不是黑匣子的组成，可以增强DNN行为的解释性。因此，如果成功的话，该项目将对这些程序员的生产率，他们所部署的DNN模型的可理解性和可维护性以及其产生的软件系统的可扩展性和正确性产生重大积极影响。其他影响还将包括：基于研究的高级培训以及未来计算机科学家的实验和系统建设专业知识的增强，将研究结果纳入爱荷华州立大学的课程，并促进与模块化研究相关的主题的整合，以及通过基于研究的培训的参与来培训的机会。基金会的智力优点和更广泛的影响评论标准。

项目成果

Mutation-based Fault Localization of Deep Neural Networks

• DOI: 10.1109/ase56229.2023.00171
• 发表时间: 2023-09
• 期刊: 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)
• 作者: Ali Ghanbari;Deepak-George Thomas;Muhammad Arbab Arshad;Hridesh Rajan

What kinds of contracts do ML APIs need?

• DOI: 10.1007/s10664-023-10320-z
• 发表时间: 2023-07
• 期刊: Empirical Software Engineering
• 影响因子: 4.1
• 作者: S. K. Samantha;Shibbir Ahmed;S. Imtiaz;Hridesh Rajan;G. Leavens

Fix Fairness, Don’t Ruin Accuracy: Performance Aware Fairness Repair using AutoML

• DOI: 10.1145/3611643.3616257
• 发表时间: 2023-06
• 期刊: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
• 作者: Giang Nguyen-;Sumon Biswas;Hridesh Rajan

Fairify: Fairness Verification of Neural Networks

• DOI: 10.1109/icse48619.2023.00134
• 发表时间: 2022-12
• 期刊: 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)
• 作者: Sumon Biswas;Hridesh Rajan

Decomposing a Recurrent Neural Network into Modules for Enabling Reusability and Replacement

• DOI: 10.1109/icse48619.2023.00093
• 发表时间: 2022-12
• 期刊: 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)
• 作者: S. Imtiaz;Fraol Batole;Astha Singh;Rangeet Pan;Breno Dantas Cruz;Hridesh Rajan