On-Device Partial Learning Technique of Convolutional Neural Network for New Classes
On-Device Partial Learning Technique of Convolutional Neural Network for New Classes
复制标题
DOI：
10.1007/s11265-020-01520-7
复制
发表时间：
2020-01-30
影响因子：
1.8
通讯作者：
Kang, Sanggil
中科院分区：
文献类型：
--
作者：
Hur, Cheonghwan;Kang, Sanggil
通常，卷积神经网络(CNN)具有复杂的网络结构，由重层组成，具有大量的参数，如卷积、池化、重新激活和全连通层。由于CNN的复杂性和计算量，CNN是在云环境中训练的。在云上学习和执行存在个人信息安全问题和通信状态依赖等问题。最近，为了缓解这两个缺点，人们在移动设备上直接对CNN进行训练。由于移动设备的资源限制，需要对CNN的结构进行压缩或减少训练开销。在本文中，我们提出了一种设备上的部分学习技术，具有以下优点：(1)不需要额外的神经网络结构，(2)减少了不必要的计算开销。我们从训练好的网络中选择影响权值的一个子集来适应新的分类类别。选择是基于每个权重对产出的贡献信息进行的，该信息是使用熵概念来衡量的。在实验部分，我们使用美国国家标准与技术研究所的混合图像数据和微软上下文中的共同目标数据两个数据集，用CNN图像分类器对我们的方法进行了演示和分析。结果，在LeNet-5和AlexNet上的计算资源的性能分别提高了1.7倍和2.3倍，内存资源的性能分别提高了1.4倍和1.6倍。
In general, Convolutional Neural Networks (CNNs) have a complex network structure consisted of heavy layers with huge number of parameters such as the convolutional, pooling, relu-activation, and fully-connected layers. Due to the complexity and computation load, CNNs are trained on a cloud environment. There are a couple of drawbacks on learning and performing on the cloud such as security problem of personal information and dependency of communication state. Recently, CNNs are directly trained at the mobile devices in order to alleviate those two drawbacks. Due to the resource limitation of the mobile devices, the structure of CNNs needs to be compressed or to reduce training overhead. In this paper, we propose an on-device partial learning technique with the following benefits: (1) does not require additional neural network structures, and (2) reduces unnecessary computation overhead. We select a subset of influential weights from a trained network to accommodate the new classification class. The selection is made based on the information of the contribution of each weight to output, which is measured using the entropy concept. In the experimental section, we demonstrate and analyze our method with a CNN image classifier using two datasets such as Mixed National Institute of Standards and Technology image data and Microsoft Common Objection in Context data. As a result, the computational resources at LeNet-5 and AlexNet showed performance improvements of 1.7x and 2.3x, respectively, and memory resources demonstrated performance improvements of 1.4x and 1.6x, respectively.