CAREER: An Embodied Intelligence Approach to Neural Architecture Search

机器学习(ML)模型的最新进展，如深度神经网络，在解决从健康和健康到环境科学到国防等广泛领域的问题方面显示出巨大的前景。然而，这些方法的实际应用目前需要大量的ML训练和经验来实施，因为它们对非直观和复杂的配置设置(如深度神经网络的大小和形状)非常敏感。自动机器学习(AutoML)的子领域试图通过创建根据给定问题的需求自动自我配置的模型和管道来帮助降低进入门槛。在AutoML中，神经体系结构搜索(NAS)旨在自动找到深层神经网络的理想结构。使用ML找到深度神经网络的最优形状和形式的过程大致类似于在生物生物中创建形状和形状的经过充分研究的进化和发展过程，或者在进化机器人领域自动找到机器人的形状和形状的过程。尽管这些子领域之间有相似之处，AutoML中的工作可能会对我们驾驭数据革命并在广泛的应用领域产生实际影响的能力产生巨大影响，但很少有神经体系结构搜索算法的例子受到具体化机器人和动物发展的方法、成功和挑战的启发。在这项工作中，我们建议：(1)系统地比较不同神经网络结构的质量，通过分析它们的网络拓扑中的“体现智能”的数量。(2)强调了当前NAS“权重分担”方法的一个缺陷，并展示了脑-体联合优化的体现视角如何改进对高质量神经结构的搜索。(3)演示在整个训练过程中生长和修剪其结构的神经结构与静态神经网络结构相比有何不同。(4)创建基础设施，以便更容易地将针对真实世界问题数据集的协作整合到佛蒙特大学的机器学习和数据科学教学中。(5)创建科学的交流材料，通过互动的网络可视化和生成艺术，吸引非STEM学生参与机器学习。该项目由电气、通信和网络系统部门(ECCS)和既定的激励竞争研究计划(EPSCoR)共同资助。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Recent advances in Machine Learning (ML) models like deep neural networks have shown immense promise to solve problems in a wide variety of fields, ranging from health and wellness to environmental science to national defense. Yet the practical use of these methods currently requires a great deal of ML-training and experience to implement due to their sensitivity to nonintuitive and complex configuration settings like the size and shape of deep neural networks. The subfield of Automated Machine Learning (AutoML) seeks to help reduce the barriers to entry by creating models and pipelines which automatically self-configure based on the needs of a given problem. Within AutoML, Neural Architecture Search (NAS) aims to automatically find the ideal structure of a deep neural network. The process of using ML to find the optimal shape and form of a deep neural network is roughly analogous to the well-studied evolutionary and developmental processes that create shape and form in biological creatures, or that automatically find the shape and form of robots in the field of Evolutionary Robotics. Despite the analogies between these subfields, and the outsized impact that work in AutoML may have on our ability to Harness the Data Revolution and make practical impacts across a wide variety of application areas, few examples of Neural Architecture Search algorithms have been inspired by methodologies, successes, and challenges in the evolution of development of embodied robots and animals. In this work, we propose to: (1) Systematically compare the quality of different neural network architectures by analyzing the amount of “embodied intelligence” in their network topologies. (2) Highlight a shortcoming of current “weight-sharing” approaches to NAS and demonstrate how an embodied perspective to brain-body co-optimization may improve search for high quality neural architectures. (3) Demonstrate how neural architectures that grow and prune their structures throughout training compare to static neural network architectures. (4) Create infrastructure to more easily integrate collaborations on real-world problems datasets into the teaching of machine learning and data science at the University of Vermont. (5) Create scientific communication materials for engaging non-STEM students in machine learning via interactive network visualizations and generative art. This project is jointly funded by the Electrical, Communications and Cyber Systems Division (ECCS) and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.