SHF:Small: More Modular Deep Learning

该项目将研究一类被称为深度学习的机器学习算法，该算法在学术界和工业界受到了广泛关注。深度学习有大量重要的社会应用，从自动驾驶汽车到Siri和Alexa等问答系统。深度学习算法使用多层转换函数将输入转换为输出，每层依次学习数据中更高层次的抽象。大数据集的可用性使得训练深度学习模型成为可能。由于这些层以网络的形式组织，因此这些模型也被称为深度神经网络（DNN）。虽然深度学习对软件行为的整体理解的影响还没有定论，但它在广泛领域和安全关键系统（如自动驾驶、航空系统、医疗分析等）的使用和应用的显著上升，结合在一起，保证了在深度学习的情况下对软件工程实践的研究。其中一个挑战是实现DNN部件的重用和替换，这有可能使DNN的开发更加可靠。该项目将研究一种全面的方法，系统地研究深度神经网络分解为模块，以实现这些模块的重用、替换和独立进化。模块是软件系统的独立部分，无需对系统的其余部分进行重大更改即可对其进行测试、验证或利用。允许DNN模块的重用有望减少构建DNN模型所需的能量和数据密集型训练工作。允许替换有望帮助替换DNN模型中的错误功能，而无需昂贵的再训练步骤。研究者的初步工作表明，可以将完全连接的神经网络和CNN模型分解为模块，并将模块的概念概念化。这个项目的主要目标和智力优势是在三个维度上进一步扩展这种分解方法：(1)分解方法是否可以推广到大型自然语言处理（NLP）模型中，其中二氧化碳排放量有望大幅减少？(2)将深度神经网络分解成模块应该使用什么标准？更好地理解分解标准可以帮助为dnn的设计和实现提供信息，并减少变化的影响。(3)虽然粗粒度分解在fcnn和cnn中工作得很好，但是将dnn分解成使用and - or - not原语连接的模块和结构化分解是否有可能实现更多的重用（特别是对于较大的dnn）并提供对dnn行为的更深入的见解？该项目还结合了一个严格的评估计划，使用广泛研究的数据集。该项目预计将通过为深度学习的科学和实践提供信息，对社会产生广泛影响。当前软件开发人员面临的一个严重问题是，深度学习在我们的软件系统中得到了广泛的应用，但科学家和实践者还没有清楚地处理一些关键问题，如DNN模型的可解释性、DNN重用、替换、独立测试和独立开发。显然没有必要研究模块化的概念，因为在深度学习时代之前训练的神经网络模型大多很小，在小数据集上训练，并且主要用作实验特征。该项目开发的深度神经网络模块的概念如果成功，将有助于在该领域的许多公开挑战上取得重大进展。深度神经网络模块可以在另一个环境中重用已经训练好的深度神经网络模块。将深度神经网络视为深度神经网络模块的组成，而不是一个黑匣子，可以增强深度神经网络行为的可解释性。这个项目，如果成功的话，将会对这些程序员的生产力、他们所部署的DNN模型的可理解性和可维护性，以及他们所生产的软件系统的可伸缩性和正确性产生巨大的积极影响。其他影响将包括：以研究为基础的高级培训，以及提高未来计算机科学家的实验和系统构建专业知识，将研究成果纳入爱荷华州立大学的课程，促进模块化研究相关主题的整合，以及增加代表性不足的群体参与研究培训的机会。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project will study a class of machine learning algorithms known as deep learning that has received much attention in academia and industry. Deep learning has a large number of important societal applications, from self-driving cars to question-answering systems such as Siri and Alexa. A deep learning algorithm uses multiple layers of transformation functions to convert inputs to outputs, each layer learning higher-level of abstractions in the data successively. The availability of large datasets has made it feasible to train deep learning models. Since the layers are organized in the form of a network, such models are also referred to as deep neural networks (DNN). While the jury is still out on the impact of deep learning on the overall understanding of software's behavior, a significant uptick in its usage and applications in wide-ranging areas and safety-critical systems, e.g., autonomous driving, aviation system, medical analysis, etc., combine to warrant research on software engineering practices in the presence of deep learning. One challenge is to enable the reuse and replacement of the parts of a DNN that has the potential to make DNN development more reliable. This project will investigate a comprehensive approach to systematically investigate the decomposition of deep neural networks into modules to enable reuse, replacement, and independent evolution of those modules. A module is an independent part of a software system that can be tested, validated, or utilized without a major change to the rest of the system. Allowing the reuse of DNN modules is expected to reduce energy- and data-intensive training efforts to construct DNN models. Allowing replacement is expected to help replace faulty functionality in DNN models without needing costly retraining steps. The preliminary work of the investigator has shown that it is possible to decompose fully connected neural networks and CNN models into modules and conceptualize the notion of modules. The main goals and the intellectual merits of this project are to further expand this decomposition approach along three dimensions: (1) Does the decomposition approach generalize to large Natural Language Processing (NLP) models, where a huge reduction in CO2e emission is expected? (2) What criteria should be used for decomposing a DNN into modules? A better understanding of the decomposition criteria can help inform the design and implementation of DNNs and reduce the impact of changes. (3) While coarse-grained decomposition has worked well for FCNNs and CNNs, does a finer-grained decomposition of DNNs into modules connected using AND-OR-NOT primitives a la structured decomposition has the potential to both enable more reuse (especially for larger DNNs) and provide deeper insights into the behavior of DNNs? The project also incorporates a rigorous evaluation plan using widely studied datasets. The project is expected to broadly impact society by informing the science and practice of deep learning. A serious problem facing the current software development workforce is that deep learning is widely utilized in our software systems, but scientists and practitioners do not yet have a clear handle on critical problems such as explainability of DNN models, DNN reuse, replacement, independent testing, and independent development. There was no apparent need to investigate the notions of modularity as neural network models trained before the deep learning era were mostly small, trained on small datasets, and were mostly used as experimental features. The notion of DNN modules developed by this project, if successful, could help make significant advances on a number of open challenges in this area. DNN modules could enable the reuse of already trained DNN modules in another context. Viewing a DNN as a composition of DNN modules instead of a black box could enhance the explainability of a DNN's behavior. This project, if successful, will thus have a large positive impact on the productivity of these programmers, the understandability and maintainability of the DNN models that they deploy, and the scalability and correctness of software systems that they produce. Other impacts will include: research-based advanced training as well as enhancement in experimental and system-building expertise of future computer scientists, incorporation of research results into courses at Iowa State University as well as facilitating the integration of modularity research-related topics, and increased opportunities for the participation of underrepresented groups in research-based training.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.