Choose Your Neuron: Incorporating Domain Knowledge through Neuron-Importance
Choose Your Neuron: Incorporating Domain Knowledge through Neuron-Importance
复制标题
DOI：
10.1007/978-3-030-01261-8_32
复制
发表时间：
2018-08
期刊：
影响因子：
--
通讯作者：
Ramprasaath R. Selvaraju;Prithvijit Chattopadhyay;Mohamed Elhoseiny;Tilak Sharma;Dhruv Batra;Devi Parikh-
中科院分区：
文献类型：
--
作者：
Ramprasaath R. Selvaraju;Prithvijit Chattopadhyay;Mohamed Elhoseiny;Tilak Sharma;Dhruv Batra;Devi Parikh-
卷积神经网络中的单个神经元被监督用于图像级分类任务，已经被证明可以隐式地学习语义上有意义的概念，从简单的纹理和形状到整个或部分对象，形成通过学习过程获得的概念的“字典”。在这项工作中，我们介绍了一个简单，高效的零杆学习方法的基础上，这一观察。我们的方法，我们称之为神经元重要性感知权重转移（NIWT），学习将关于新的“看不见的”类的领域知识映射到这个学习概念的字典上，然后优化网络参数，可以有效地联合收割机这些概念-本质上是通过在深度网络中发现和组合学习的语义概念来学习分类器。我们的方法在CUBirds和AWA 2广义零射击学习基准测试上比以前的方法有所改进。我们展示了我们的方法上的一组不同的语义输入外部域知识，包括属性和自然语言标题。此外，通过学习逆映射，NIWT可以为新学习的分类器所做的预测提供视觉和文本解释，并提供神经元名称。我们的代码可以在https：//github上找到。com/ramplant/neuron-importance-zsl.
Individual neurons in convolutional neural networks supervised for image-level classification tasks have been shown to implicitly learn semantically meaningful concepts ranging from simple textures and shapes to whole or partial objects–forming a “dictionary” of concepts acquired through the learning process. In this work we introduce a simple, efficient zero-shot learning approach based on this observation. Our approach, which we call Neuron Importance-Aware Weight Transfer (NIWT), learns to map domain knowledge about novel “unseen” classes onto this dictionary of learned concepts and then optimizes for network parameters that can effectively combine these concepts–essentially learning classifiers by discovering and composing learned semantic concepts in deep networks. Our approach shows improvements over previous approaches on the CUBirds and AWA2 generalized zero-shot learning benchmarks. We demonstrate our approach on a diverse set of semantic inputs as external domain knowledge including attributes and natural language captions. Moreover by learning inverse mappings, NIWT can provide visual and textual explanations for the predictions made by the newly learned classifiers and provide neuron names. Our code is available at https://github. com/ramprs/neuron-importance-zsl.