AI-Generating Algorithms: AI that improves itself by automatically creating learning challenges

创建强大的人工智能（AI），快速学习在各种领域（即通用AI）中表现良好，有可能深刻改善人类福利。我们如何才能创建如此强大的AI（机器学习/ML）系统？深度神经网络（深度学习）是各种领域中最先进的人工智能。然而，它们仍然缺乏人类水平的通用性和学习速度。我们如何缩小这一差距？占主导地位的方法，我称之为“人工智能方法”，试图逐一发现智能所需的数百个部分中的每一个。这条道路还假设我们有一天会完成一项艰巨的任务，即确定如何将所有这些碎片联合收割机组合成一台复杂的思维机器。在2019年，我提出了另一条可能更快的道路。它基于ML的明确趋势，即手工设计的解决方案最终将被更有效的学习解决方案所取代。我们的想法是研究如何创建AI生成算法（AI-GAs），这些算法可以自动学习如何生成更强大的AI系统。通过AI-GA，ML可以解决人工智能中的重大挑战，例如学习探索，估计不确定性，学习而不忘记等，并将这些部分结合在一起。 三大支柱对于在AI GA上取得进展至关重要：（1）自动搜索神经网络架构，（2）元学习学习算法本身（即学习学习），以及（3）自动生成新的，多样化的学习环境。 ML社区已经在大量研究支柱1。我的实验室有两个长期目标：（1）培训HQP，特别是代表性不足的少数民族，蓬勃发展并实现他们的目标（本提案每年培训3名博士，2名硕士和1名本科生）。(2)通过专注于深度强化学习代理的支柱2和3，我们将创建开放式流程，允许代理无限期地学习各种不同的，重要的，具有挑战性的技能。虽然完全实现AI-GA愿景需要时间，但在未来五年内，我们可以在许多方面取得重要进展。我们已经对支柱3（例如POET，Go-Explore）和支柱2（例如我们的论文“学习不断学习”）的可行性进行了高影响力的演示。ML社区也在这些方向上取得了令人兴奋的进展，在某些情况下受到我们工作的启发，产生了新的，令人兴奋的创新，我们可以在此基础上继续发展。四个短期目标（见提案）涉及我的HQP，合作者，我通过测试各种假设来进一步取得进展。 这样的研究（1）朝着更普遍、更强大的人工智能方向发展，它可以产生数不清的经济效益，并有意义地改善全世界每个人的生活质量。(2)同时也是培养（尤其是代表性不足的）科学家进行优秀、雄心勃勃、技术性强、值得信赖的科学研究的工具。

The creation of powerful artificial intelligence (AI) that quickly learns to perform well in a large variety of domains (i.e. general AI), has the potential to profoundly improve human welfare. How might we create such powerful AI (aka machine learning/ML) systems? Deep neural networks (deep learning) are state-of-the-art AI in a wide variety of domains. However, they still lack human-level generality and learning speed. How can we close that gap? The dominant approach, which I call the "manual AI approach", attempts to discover each of the hundreds of pieces required for intelligence one by one. This path also assumes we will one day complete the Herculean task of determining how to combine all of those pieces into a complex thinking machine. In 2019 I proposed another path that may be faster. It is based on the clear trend in ML that hand-designed solutions are eventually replaced by more effective, learned solutions. The idea is to do research into creating AI-generating algorithms (AI-GAs), which automatically learn how to produce ever-more powerful AI systems. With AI-GAs, ML can do the heavy lifting of solving the grand challenges in AI, such as learning to explore, estimating uncertainty, learning without forgetting, etc, and combining these pieces together. Three Pillars are essential to make progress on AI-GAs: (1) automatically searching for neural network architectures, (2) meta-learning the learning algorithms themselves (i.e. learning to learn), and (3) automatically generating new, diverse learning environments endlessly. The ML community is already heavily studying Pillar 1. My lab has two long-term goals: (1) train HQP, especially underrepresented minorities, to flourish and achieve their goals (this proposal trains 3 PhD, 2 MSc, and 1 undergrad per year). (2) make progress on AI-GAs by focusing on Pillars 2 and 3 for deep reinforcement learning agents, i.e. we will create open-ended processes that allow agents to indefinitely learn a variety of diverse, important, challenging skills. While fully realizing the AI-GA vision will take time, there are many fronts on which we can make important progress over the next five years. We have already made high-impact demonstrations of the feasibility of Pillar 3 (e.g. POET, Go-Explore) and Pillar 2 (e.g. our paper "Learning to Continually Learn"). The ML community has also made exciting advances in these directions, in some cases inspired by our work, producing new, exciting innovations we can build on. Four short-term goals (see proposal) involve my HQP, collaborators, and I building on this momentum by testing a variety of hypotheses for how to make further progress. Such research (1) moves toward more general, powerful AI, which can yield untold economic benefits and meaningfully improve the quality of life for every human worldwide. (2) doubles as a vehicle to train (especially underrepresented) scientists to conduct excellent, ambitious, technical, trustworthy science.