AI-Generating Algorithms: AI that improves itself by automatically creating learning challenges

点击翻译按钮获取中文摘要

The creation of powerful artificial intelligence (AI) that quickly learns to perform well in a large variety of domains (i.e. general AI), has the potential to profoundly improve human welfare. How might we create such powerful AI (aka machine learning/ML) systems? Deep neural networks (deep learning) are state-of-the-art AI in a wide variety of domains. However, they still lack human-level generality and learning speed. How can we close that gap? The dominant approach, which I call the "manual AI approach", attempts to discover each of the hundreds of pieces required for intelligence one by one. This path also assumes we will one day complete the Herculean task of determining how to combine all of those pieces into a complex thinking machine. In 2019 I proposed another path that may be faster. It is based on the clear trend in ML that hand-designed solutions are eventually replaced by more effective, learned solutions. The idea is to do research into creating AI-generating algorithms (AI-GAs), which automatically learn how to produce ever-more powerful AI systems. With AI-GAs, ML can do the heavy lifting of solving the grand challenges in AI, such as learning to explore, estimating uncertainty, learning without forgetting, etc, and combining these pieces together. Three Pillars are essential to make progress on AI-GAs: (1) automatically searching for neural network architectures, (2) meta-learning the learning algorithms themselves (i.e. learning to learn), and (3) automatically generating new, diverse learning environments endlessly. The ML community is already heavily studying Pillar 1. My lab has two long-term goals: (1) train HQP, especially underrepresented minorities, to flourish and achieve their goals (this proposal trains 3 PhD, 2 MSc, and 1 undergrad per year). (2) make progress on AI-GAs by focusing on Pillars 2 and 3 for deep reinforcement learning agents, i.e. we will create open-ended processes that allow agents to indefinitely learn a variety of diverse, important, challenging skills. While fully realizing the AI-GA vision will take time, there are many fronts on which we can make important progress over the next five years. We have already made high-impact demonstrations of the feasibility of Pillar 3 (e.g. POET, Go-Explore) and Pillar 2 (e.g. our paper "Learning to Continually Learn"). The ML community has also made exciting advances in these directions, in some cases inspired by our work, producing new, exciting innovations we can build on. Four short-term goals (see proposal) involve my HQP, collaborators, and I building on this momentum by testing a variety of hypotheses for how to make further progress. Such research (1) moves toward more general, powerful AI, which can yield untold economic benefits and meaningfully improve the quality of life for every human worldwide. (2) doubles as a vehicle to train (especially underrepresented) scientists to conduct excellent, ambitious, technical, trustworthy science.