Learning deep neural network architectures for novel data domains

点击翻译按钮获取中文摘要

Deep Neural Networks (DNNs) are the technology behind many of the contemporary Artificial Intelligence (AI) solutions allowing computers to understand photos (Computer Vision), smart assistants to understand our spoken questions (Speech), and translate a foreign language (Natural Language Processing). Much of the success of DNNs in these applications can be attributed to the data domain-specific DNN architectures which allow DNNs to more easily learn compact and meaningful representations within specific data domains. These architectures allow DNNs to be more data efficient, i.e. require less training data to learn to solve a problem, and also increase generalization, i.e. the ability of these representations to perform well for previously unseen data. Examples of such DNN architectures include such as Convolutional Neural Networks (CNNs) for natural images or Transformers for natural language. These architectures were hand-designed using years of research and development to incorporate domain-specific knowledge in the architecture (i.e. structure) of neural network models. Unfortunately applying DNNs to problems within data domains which are poorly addressed by current DNN architectures is fundamentally limited. This includes much of the data collected in industrial informatics, specifically in the agricultural, finance and energy industries. Creating better DNN architectures for these data domains, and problems within these data domains, is expensive, requiring years of research and significant domain and machine learning expertise. We propose instead to explore the potential for state-of-the-art methods of training sparse neural networks to learn DNN structure from training data, specifically in poorly addressed data domains.