AI-Generating Algorithms: AI that improves itself by automatically creating learning challenges

强大的人工智能(AI)的创建，迅速学会在许多领域表现良好(即一般人工智能)，具有深刻改善人类福利的潜力。我们如何才能创建如此强大的人工智能(又名机器学习/ML)系统？深度神经网络(深度学习)是人工智能在各个领域的前沿。然而，它们仍然缺乏人类水平的通用性和学习速度。我们如何才能缩小这一差距？占主导地位的方法，我称之为“人工智能方法”，试图逐一发现智能所需的数百件东西中的每一件。这条道路还假设我们有一天会完成一项艰巨的任务，即确定如何将所有这些片段组合成一台复杂的思考机器。2019年，我提出了另一条可能更快的道路。基于ML的明显趋势，手工设计的解决方案最终将被更有效的、可学习的解决方案所取代。这个想法是为了研究如何创建人工智能生成算法(AI-GAS)，该算法自动学习如何生成功能越来越强大的人工智能系统。有了AI-GAS，ML可以做解决AI中重大挑战的重担，比如学习探索、估计不确定性、学习而不忘等，并将这些部件组合在一起。AI-GAS取得进展的三个支柱是：(1)自动搜索神经网络结构，(2)元学习算法本身(即学习学习)，(3)无休止地自动生成新的、多样化的学习环境。ML社区已经在大力研究支柱1。我的实验室有两个长期目标：(1)培养HQP，特别是代表不足的少数族裔，以蓬勃发展并实现他们的目标(这项提议每年培养3名博士、2名硕士和1名本科生)。(2)通过专注于深度强化学习代理的支柱2和3，在AI-GAS方面取得进展，即我们将创建开放式过程，允许代理无限期地学习各种不同的、重要的、具有挑战性的技能。虽然完全实现AI-GA愿景需要时间，但在未来五年，我们可以在许多方面取得重要进展。我们已经对支柱3(如PEAT、GO-EXPLOVER)和支柱2(如我们的论文《学会持续学习》)的可行性进行了高影响力的演示。ML社区也在这些方向上取得了令人兴奋的进展，在某些情况下受到我们工作的启发，产生了我们可以建立在其上的新的、令人兴奋的创新。四个短期目标(见提案)涉及我的HQP、合作者和我通过测试如何取得进一步进展的各种假设来建立这种势头。这样的研究(1)走向更普遍、更强大的人工智能，它可以产生无数的经济利益，并有意义地改善世界各地每个人的生活质量。(2)作为一种工具，培训(特别是代表不足的)科学家从事优秀、雄心勃勃、技术可靠的科学研究。

The creation of powerful artificial intelligence (AI) that quickly learns to perform well in a large variety of domains (i.e. general AI), has the potential to profoundly improve human welfare. How might we create such powerful AI (aka machine learning/ML) systems? Deep neural networks (deep learning) are state-of-the-art AI in a wide variety of domains. However, they still lack human-level generality and learning speed. How can we close that gap? The dominant approach, which I call the "manual AI approach", attempts to discover each of the hundreds of pieces required for intelligence one by one. This path also assumes we will one day complete the Herculean task of determining how to combine all of those pieces into a complex thinking machine. In 2019 I proposed another path that may be faster. It is based on the clear trend in ML that hand-designed solutions are eventually replaced by more effective, learned solutions. The idea is to do research into creating AI-generating algorithms (AI-GAs), which automatically learn how to produce ever-more powerful AI systems. With AI-GAs, ML can do the heavy lifting of solving the grand challenges in AI, such as learning to explore, estimating uncertainty, learning without forgetting, etc, and combining these pieces together. Three Pillars are essential to make progress on AI-GAs: (1) automatically searching for neural network architectures, (2) meta-learning the learning algorithms themselves (i.e. learning to learn), and (3) automatically generating new, diverse learning environments endlessly. The ML community is already heavily studying Pillar 1. My lab has two long-term goals: (1) train HQP, especially underrepresented minorities, to flourish and achieve their goals (this proposal trains 3 PhD, 2 MSc, and 1 undergrad per year). (2) make progress on AI-GAs by focusing on Pillars 2 and 3 for deep reinforcement learning agents, i.e. we will create open-ended processes that allow agents to indefinitely learn a variety of diverse, important, challenging skills. While fully realizing the AI-GA vision will take time, there are many fronts on which we can make important progress over the next five years. We have already made high-impact demonstrations of the feasibility of Pillar 3 (e.g. POET, Go-Explore) and Pillar 2 (e.g. our paper "Learning to Continually Learn"). The ML community has also made exciting advances in these directions, in some cases inspired by our work, producing new, exciting innovations we can build on. Four short-term goals (see proposal) involve my HQP, collaborators, and I building on this momentum by testing a variety of hypotheses for how to make further progress. Such research (1) moves toward more general, powerful AI, which can yield untold economic benefits and meaningfully improve the quality of life for every human worldwide. (2) doubles as a vehicle to train (especially underrepresented) scientists to conduct excellent, ambitious, technical, trustworthy science.