CAREER: Foundations of Next-Generation Neural Architecture Search

深度学习在许多重要问题上取得了显著的人工智能突破，如自动驾驶车辆的对象识别、声控助手和自动机器翻译。这些突破的核心是复杂的、特定于领域的深度神经网络结构的设计。然而，只有少数训练有素的研究人员配备了资源和专业知识来承担这一艰巨的临时设计过程。此外，设计工作在很大程度上局限于少数领域的应用，最明显的是计算机视觉和自然语言处理。虽然新兴的神经体系结构搜索(NAS)领域旨在使神经网络体系结构的设计自动化，但到目前为止，NAS的现有工作仅局限于这些研究得很好的领域。该项目旨在开发、分析和实现能够超越这些受限领域的自动化体系结构设计的新方法。该项目涉及与新领域的实践者合作，以使他们能够为其应用程序开发新的体系结构。该项目还将包括广泛的教育努力，以创建一门涵盖机器学习工作流程整个生命周期的新课程，包括广泛处理神经网络的自动化设计和调整。课程材料将免费分发，以便于在全球范围内采用，并适用于为高中生创建一个短期课程。该项目的重点是开发原则性、高效和自动化的神经体系结构搜索(NAS)功能，使从业者能够无缝地为新问题创建新的体系结构。为了实现这一目标，该项目提出了一种全新的NAS范式，由NAS的两个核心组件共同设计驱动，即架构搜索空间和在这些空间中搜索的方法。研究人员将展示他们提出的技术在许多领域的有效性，包括那些不存在专家设计的体系结构的领域。正在解决的技术问题融合了优化、学习理论、信号处理和机器学习系统的思想，并与压缩感知、弱监督和元学习问题联系在一起。拟议的NAS工作将在释放新的深度学习应用在新领域的潜力方面发挥变革性的作用，并将为研究生提供培训机会。此外，该项目拟议的活动通过以下方式强调可访问性和广泛传播：向世界各地的数据科学家传播基础教育材料；在机器学习系统研究社区中增长和促进多样性；以及通过开源活动、对卡内基梅隆大学的机器学习博客的贡献和反复出现的播客来广泛采用行业。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Deep learning has led to remarkable artificial intelligence breakthroughs on many important problems such as object recognition for autonomous vehicles, voice-activated assistants, and automated machine translation. At the heart of these breakthroughs is the design of complex, domain-specific deep neural network architectures. However, only a small set of highly-trained researchers are equipped with the resources and expertise to undertake this arduous, ad-hoc design process. Moreover, design efforts have been largely limited to applications in a handful of domains, most notably computer vision and natural language processing. While the burgeoning field of neural architecture search (NAS) aims to automate the design of neural network architectures, existing work on NAS has to date narrowly focused on these same well-studied domains. This project aims to develop, analyze, and implement novel methods that enable automated architecture design beyond these restricted domains. The project involves collaborations with practitioners in new domains to empower them to develop new architectures for their applications. The project will also include extensive educational efforts to create a new course that covers the complete lifecycle of machine learning workflows, including extensive treatment on the automated design and tuning of neural networks. The course material will be freely distributed to facilitate worldwide adoption and adapted to create a short course for high school students.The focus of this project is to develop principled, efficient, and automated Neural Architecture Search (NAS) capabilities to enable practitioners to seamlessly create novel architectures for new problems. To achieve this goal, the project proposes a fundamentally new NAS paradigm driven by the co-design of the two core components of NAS, namely architecture search spaces and methods to search through these spaces. The researchers will demonstrate the effectiveness of their proposed techniques across numerous domains, including those where expert-designed architectures do not exist. The technical problems being tackled blend ideas from optimization, learning theory, signal processing, and machine learning systems, and draw connections to the problems of compressed sensing, weak supervision, and meta-learning. The proposed NAS work will be transformational in unlocking the potential of novel deep learning applications in new domains, and will provide training opportunities for graduate students. Moreover, the project's proposed activities emphasize accessibility and broad dissemination via: foundational educational material disseminated to data scientists worldwide; growing and promoting diversity in the Machine Learning Systems research community; and widespread industry adoption via open-source activities, contributions to Carnegie Mellon University's Machine Learning blog, and a recurring podcast.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.