Learning deep neural network architectures for novel data domains

深度神经网络（DNN）是许多当代人工智能（AI）解决方案背后的技术，允许计算机理解照片（计算机视觉），智能助手理解我们的口语问题（语音），并翻译外语（自然语言处理）。DNN在这些应用中的大部分成功可以归功于特定于数据域的DNN架构，该架构允许DNN更容易地学习特定数据域中的紧凑和有意义的表示。这些架构允许DNN具有更高的数据效率，即需要更少的训练数据来学习解决问题，并且还提高了泛化能力，即这些表示能够很好地处理以前看不见的数据。这种DNN架构的示例包括用于自然图像的卷积神经网络（CNN）或用于自然语言的变换器。这些架构是使用多年的研究和开发手工设计的，以将特定领域的知识纳入神经网络模型的架构（即结构）中。不幸的是，将DNN应用于当前DNN架构无法解决的数据域中的问题从根本上受到限制。这包括工业信息学收集的大部分数据，特别是农业、金融和能源行业的数据。为这些数据域以及这些数据域中的问题创建更好的DNN架构是昂贵的，需要多年的研究以及重要的领域和机器学习专业知识。相反，我们建议探索训练稀疏神经网络的最先进方法的潜力，以从训练数据中学习DNN结构，特别是在处理不良的数据域中。

Deep Neural Networks (DNNs) are the technology behind many of the contemporary Artificial Intelligence (AI) solutions allowing computers to understand photos (Computer Vision), smart assistants to understand our spoken questions (Speech), and translate a foreign language (Natural Language Processing). Much of the success of DNNs in these applications can be attributed to the data domain-specific DNN architectures which allow DNNs to more easily learn compact and meaningful representations within specific data domains. These architectures allow DNNs to be more data efficient, i.e. require less training data to learn to solve a problem, and also increase generalization, i.e. the ability of these representations to perform well for previously unseen data. Examples of such DNN architectures include such as Convolutional Neural Networks (CNNs) for natural images or Transformers for natural language. These architectures were hand-designed using years of research and development to incorporate domain-specific knowledge in the architecture (i.e. structure) of neural network models. Unfortunately applying DNNs to problems within data domains which are poorly addressed by current DNN architectures is fundamentally limited. This includes much of the data collected in industrial informatics, specifically in the agricultural, finance and energy industries. Creating better DNN architectures for these data domains, and problems within these data domains, is expensive, requiring years of research and significant domain and machine learning expertise. We propose instead to explore the potential for state-of-the-art methods of training sparse neural networks to learn DNN structure from training data, specifically in poorly addressed data domains.