Learn Basic Skills and Reuse: Modularized Adaptive Neural Architecture Search (MANAS)
Learn Basic Skills and Reuse: Modularized Adaptive Neural Architecture Search (MANAS)
复制标题
DOI：
10.1145/3511808.3557385
复制
发表时间：
2022-08
期刊：
影响因子：
--
通讯作者：
H. Chen;Yunqi Li;He Zhu;Yongfeng Zhang
中科院分区：
文献类型：
--
作者：
H. Chen;Yunqi Li;He Zhu;Yongfeng Zhang
人类智力能够首先学习一些解决基本问题的基本技能，然后将这些基本技能组合成解决复杂或新问题的复杂技能。例如，基本功“挖坑”、“放树”、“回填”和“浇水”组成了一个复杂的技能“种树”。此外，一些基本功可以重复用于解决其他问题。例如，基本功“挖坑”不仅可以用来种树，还可以用来挖宝藏、修筑排水沟或填埋。学习基本技能并将其重复用于各种任务的能力对人类非常重要，因为这有助于避免为解决每个单独的任务而学习太多技能，并使仅通过学习几个基本技能就可以解决组成数量的任务成为可能，这节省了人脑中相当大的内存和计算能力。我们认为，机器智能还应该捕捉学习基本技能的能力，并通过组合成复杂的技能来重用它们。在计算机科学语言中，每一项基本技能都是一个“模块”，它是一个具有具体含义、执行具体基本操作的可重复使用的网络。这些模块被组装成一个更大的“模型”，用于完成更复杂的任务。装配过程对输入或任务是自适应的，即对于给定的任务，模块应该被组装成最适合于求解给定任务的模型。因此，不同的输入/任务可能具有不同的组装模型。在这项工作中，我们以推荐系统为例，提出了模块化自适应神经结构搜索(MANAS)来演示上述思想。神经结构搜索(NAS)已经显示出它在发现高级神经结构方面的强大能力。然而，现有的NAS大多专注于搜索全局体系结构，而不考虑具体的输入，即体系结构不适应输入。在这项工作中，我们借鉴了模块化神经逻辑推理的思想，考虑了三个基本的逻辑运算模块：与、或、非。同时，为每个用户提供推荐被视为一项任务。MANAS自动将逻辑操作模块组装成为给定用户量身定做的网络结构。结果，为每个用户组装了个性化的神经体系结构，以向用户提供推荐，这意味着得到的神经体系结构与模型的输入(即，用户的过去行为)是自适应的。在不同数据集上的实验表明，MANAS组装的自适应体系结构的性能优于静态全局体系结构。进一步的实验和实证分析为MANAS的有效性提供了见解。该代码在https://github.com/TalonCB/MANAS.上是开源的
Human intelligence is able to first learn some basic skills for solving basic problems and then assemble such basic skills into complex skills for solving complex or new problems. For example, the basic skills "dig hole,'' "put tree,'' "backfill'' and "watering'' compose a complex skill "plant a tree''. Besides, some basic skills can be reused for solving other problems. For example, the basic skill "dig hole'' not only can be used for planting a tree, but also can be used for mining treasures, building a drain, or landfilling. The ability to learn basic skills and reuse them for various tasks is very important for humans because it helps to avoid learning too many skills for solving each individual task, and makes it possible to solve a compositional number of tasks by learning just a few number of basic skills, which saves a considerable amount of memory and computational power in the human brain. We believe that machine intelligence should also capture the ability of learning basic skills and reusing them by composing into complex skills. In computer science language, each basic skill is a "module'', which is a reusable network that has a concrete meaning and performs a concrete basic operation. The modules are assembled into a bigger "model'' for doing a more complex task. The assembling procedure is adaptive to the input or task, i.e., for a given task, the modules should be assembled into the most suitable model for solving the given task. As a result, different inputs/tasks could have different assembled models. In this work, we take recommender system as an example and propose Modularized Adaptive Neural Architecture Search (MANAS) to demonstrate the above idea. Neural Architecture Search (NAS) has shown its power in discovering superior neural architectures. However, existing NAS mostly focus on searching for a global architecture regardless of the specific input, i.e., the architecture is not adaptive to the input. In this work, we borrow the idea from modularized neural logic reasoning and consider three basic logical operation modules: AND, OR, NOT. Meanwhile, making recommendations for each user is considered as a task. MANAS automatically assembles the logical operation modules into a network architecture tailored for the given user. As a result, a personalized neural architecture is assembled for each user to make recommendations for the user, which means that the resulting neural architecture is adaptive to the model's input (i.e., the user's past behaviors). Experiments on different datasets show that the adaptive architecture assembled by MANAS outperforms static global architectures. Further experiments and empirical analysis provide insights to the effectiveness of MANAS. The code is open-source at https://github.com/TalonCB/MANAS.