CAREER: Foundations of Next-Generation Neural Architecture Search

深度学习在许多重要问题上带来了显著的人工智能突破，例如自动驾驶汽车的物体识别，语音激活助手和自动机器翻译。这些突破的核心是设计复杂的、特定领域的深度神经网络架构。然而，只有一小部分训练有素的研究人员具备资源和专业知识来承担这一艰巨的特设设计过程。 此外，设计工作在很大程度上仅限于少数几个领域的应用，最明显的是计算机视觉和自然语言处理。虽然新兴的神经架构搜索（NAS）领域旨在自动化神经网络架构的设计，但迄今为止，NAS的现有工作仅限于这些相同的研究领域。该项目旨在开发，分析和实现新的方法，使自动化架构设计超越这些限制的领域。该项目涉及与新领域的从业者合作，使他们能够为其应用程序开发新的架构。该项目还将包括广泛的教育工作，以创建一个新的课程，涵盖机器学习工作流程的完整生命周期，包括对神经网络的自动化设计和调整的广泛处理。课程材料将免费分发，以促进全球范围内的采用，并改编为高中生创建一个短期课程。该项目的重点是开发原则性，高效和自动化的神经架构搜索（NAS）功能，使从业者能够无缝地创建新的架构新的问题。为了实现这一目标，该项目提出了一个全新的NAS范式，由NAS的两个核心组件（即架构搜索空间和搜索这些空间的方法）的协同设计驱动。研究人员将展示他们提出的技术在众多领域的有效性，包括那些专家设计的架构不存在的领域。正在解决的技术问题融合了优化、学习理论、信号处理和机器学习系统的思想，并与压缩传感、弱监督和元学习的问题相联系。拟议的NAS工作将在新领域释放新型深度学习应用程序的潜力方面具有变革性，并将为研究生提供培训机会。 此外，该项目的拟议活动强调可访问性和广泛传播：向世界各地的数据科学家传播基础教育材料;增加和促进机器学习系统研究界的多样性;通过开源活动，卡内基梅隆大学机器学习博客的贡献，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Deep learning has led to remarkable artificial intelligence breakthroughs on many important problems such as object recognition for autonomous vehicles, voice-activated assistants, and automated machine translation. At the heart of these breakthroughs is the design of complex, domain-specific deep neural network architectures. However, only a small set of highly-trained researchers are equipped with the resources and expertise to undertake this arduous, ad-hoc design process. Moreover, design efforts have been largely limited to applications in a handful of domains, most notably computer vision and natural language processing. While the burgeoning field of neural architecture search (NAS) aims to automate the design of neural network architectures, existing work on NAS has to date narrowly focused on these same well-studied domains. This project aims to develop, analyze, and implement novel methods that enable automated architecture design beyond these restricted domains. The project involves collaborations with practitioners in new domains to empower them to develop new architectures for their applications. The project will also include extensive educational efforts to create a new course that covers the complete lifecycle of machine learning workflows, including extensive treatment on the automated design and tuning of neural networks. The course material will be freely distributed to facilitate worldwide adoption and adapted to create a short course for high school students.The focus of this project is to develop principled, efficient, and automated Neural Architecture Search (NAS) capabilities to enable practitioners to seamlessly create novel architectures for new problems. To achieve this goal, the project proposes a fundamentally new NAS paradigm driven by the co-design of the two core components of NAS, namely architecture search spaces and methods to search through these spaces. The researchers will demonstrate the effectiveness of their proposed techniques across numerous domains, including those where expert-designed architectures do not exist. The technical problems being tackled blend ideas from optimization, learning theory, signal processing, and machine learning systems, and draw connections to the problems of compressed sensing, weak supervision, and meta-learning. The proposed NAS work will be transformational in unlocking the potential of novel deep learning applications in new domains, and will provide training opportunities for graduate students. Moreover, the project's proposed activities emphasize accessibility and broad dissemination via: foundational educational material disseminated to data scientists worldwide; growing and promoting diversity in the Machine Learning Systems research community; and widespread industry adoption via open-source activities, contributions to Carnegie Mellon University's Machine Learning blog, and a recurring podcast.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.