CAREER: An Embodied Intelligence Approach to Neural Architecture Search

机器学习（ML）模型的最新进展，如深度神经网络，已经显示出解决各种领域问题的巨大希望，从健康和保健到环境科学再到国防。然而，这些方法的实际应用目前需要大量的机器学习训练和经验来实现，因为它们对非直观和复杂的配置设置（如深度神经网络的大小和形状）很敏感。自动化机器学习（AutoML）的子领域旨在通过创建基于给定问题的需求自动自配置的模型和管道来帮助减少进入障碍。在AutoML中，Neural Architecture Search （NAS）旨在自动找到深度神经网络的理想结构。使用ML找到深度神经网络的最佳形状和形式的过程大致类似于在生物生物中创建形状和形式的充分研究的进化和发展过程，或者在进化机器人领域自动找到机器人的形状和形式。尽管这些子领域之间存在相似之处，并且AutoML中的工作可能会对我们利用数据革命的能力产生巨大影响，并在各种应用领域产生实际影响，但很少有神经架构搜索算法的例子受到嵌入式机器人和动物发展的方法、成功和挑战的启发。在这项工作中，我们建议：(1)通过分析其网络拓扑中“具具智能”的数量，系统地比较不同神经网络架构的质量。(2)强调当前NAS“权重共享”方法的不足，并展示脑-体协同优化的具身视角如何改善对高质量神经结构的搜索。(3)演示在整个训练过程中生长和修剪其结构的神经网络架构如何与静态神经网络架构进行比较。(4)创建基础设施，以便更轻松地将现实问题数据集的协作整合到佛蒙特大学的机器学习和数据科学教学中。(5)通过交互式网络可视化和生成艺术，为非stem学生制作科学交流材料，让他们参与机器学习。该项目由电气、通信和网络系统部（ECCS）和促进竞争研究的既定计划（EPSCoR）共同资助。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recent advances in Machine Learning (ML) models like deep neural networks have shown immense promise to solve problems in a wide variety of fields, ranging from health and wellness to environmental science to national defense. Yet the practical use of these methods currently requires a great deal of ML-training and experience to implement due to their sensitivity to nonintuitive and complex configuration settings like the size and shape of deep neural networks. The subfield of Automated Machine Learning (AutoML) seeks to help reduce the barriers to entry by creating models and pipelines which automatically self-configure based on the needs of a given problem. Within AutoML, Neural Architecture Search (NAS) aims to automatically find the ideal structure of a deep neural network. The process of using ML to find the optimal shape and form of a deep neural network is roughly analogous to the well-studied evolutionary and developmental processes that create shape and form in biological creatures, or that automatically find the shape and form of robots in the field of Evolutionary Robotics. Despite the analogies between these subfields, and the outsized impact that work in AutoML may have on our ability to Harness the Data Revolution and make practical impacts across a wide variety of application areas, few examples of Neural Architecture Search algorithms have been inspired by methodologies, successes, and challenges in the evolution of development of embodied robots and animals. In this work, we propose to: (1) Systematically compare the quality of different neural network architectures by analyzing the amount of “embodied intelligence” in their network topologies. (2) Highlight a shortcoming of current “weight-sharing” approaches to NAS and demonstrate how an embodied perspective to brain-body co-optimization may improve search for high quality neural architectures. (3) Demonstrate how neural architectures that grow and prune their structures throughout training compare to static neural network architectures. (4) Create infrastructure to more easily integrate collaborations on real-world problems datasets into the teaching of machine learning and data science at the University of Vermont. (5) Create scientific communication materials for engaging non-STEM students in machine learning via interactive network visualizations and generative art. This project is jointly funded by the Electrical, Communications and Cyber Systems Division (ECCS) and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.