Resource-Efficient Convolutional Networks: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques
Resource-Efficient Convolutional Networks: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques
复制标题
DOI：
10.1145/3587095
复制
发表时间：
2021-12
影响因子：
16.6
通讯作者：
JunKyu Lee;L. Mukhanov;A. S. Molahosseini;U. Minhas;Yang Hua;Jesus Martinez del Rincon;K. Dichev
中科院分区：
文献类型：
--
作者：
JunKyu Lee;L. Mukhanov;A. S. Molahosseini;U. Minhas;Yang Hua;Jesus Martinez del Rincon;K. Dichev
卷积神经网络（CNN）用于我们的日常生活中，包括自动驾驶汽车、虚拟助手、社交网络服务、医疗保健服务和人脸识别等。然而，深度CNN在训练和推理过程中需要大量的计算资源。机器学习社区主要关注模型级优化，例如CNN的架构压缩，而系统社区则专注于实现级优化。在此期间，在算术社区中已经提出了各种算术级优化技术。本文从模型级、算法级和实现级技术三个方面对资源高效的CNN技术进行了调查，并确定了资源高效的CNN技术在三个不同级别技术中的研究差距。我们的调查根据我们的资源效率指标定义阐明了从高级技术到低级技术的影响，并讨论了资源效率CNN研究的未来趋势。
Convolutional neural networks (CNNs) are used in our daily life, including self-driving cars, virtual assistants, social network services, healthcare services, and face recognition, among others. However, deep CNNs demand substantial compute resources during training and inference. The machine learning community has mainly focused on model-level optimizations such as architectural compression of CNNs, whereas the system community has focused on implementation-level optimization. In between, various arithmetic-level optimization techniques have been proposed in the arithmetic community. This article provides a survey on resource-efficient CNN techniques in terms of model-, arithmetic-, and implementation-level techniques, and identifies the research gaps for resource-efficient CNN techniques across the three different level techniques. Our survey clarifies the influence from higher- to lower-level techniques based on our resource efficiency metric definition and discusses the future trend for resource-efficient CNN research.