Heuristic rank selection with progressively searching tensor ring network
Heuristic rank selection with progressively searching tensor ring network
复制标题
DOI：
10.1007/s40747-021-00308-x
复制
发表时间：
2021-03-17
影响因子：
5.8
通讯作者：
Xu, Zenglin
中科院分区：
文献类型：
--
作者：
Li, Nannan;Pan, Yu;Xu, Zenglin
最近，张量环网络（TRN）已经应用于深度网络，在压缩比和准确性方面取得了显著的成功。虽然高度相关的TRN的性能，秩选择很少在以前的工作中研究，通常设置为相等的实验。同时，没有任何启发式的方法来选择的排名，和枚举的方式来找到合适的排名是非常耗时的。有趣的是，我们发现部分排名元素是敏感的，通常聚集在一个狭窄的区域，即感兴趣的区域。因此，基于上述现象，我们提出了一种新的渐进式遗传算法称为渐进搜索张量环网络搜索（PSTRN），它有能力找到最佳的排名准确和有效的。通过进化阶段和渐进阶段，PSTRN可以快速收敛到感兴趣的区域，并获得良好的性能。实验结果表明，与枚举法相比，PSTRN能显著降低求秩的复杂度。此外，我们的方法在MNIST，CIFAR 10/100，UCF 11和HMDB 51等公共基准上进行了验证，达到了最先进的性能。
Recently, tensor ring networks (TRNs) have been applied in deep networks, achieving remarkable successes in compression ratio and accuracy. Although highly related to the performance of TRNs, rank selection is seldom studied in previous works and usually set to equal in experiments. Meanwhile, there is not any heuristic method to choose the rank, and an enumerating way to find appropriate rank is extremely time-consuming. Interestingly, we discover that part of the rank elements is sensitive and usually aggregate in a narrow region, namely an interest region. Therefore, based on the above phenomenon, we propose a novel progressive genetic algorithm named progressively searching tensor ring network search (PSTRN), which has the ability to find optimal rank precisely and efficiently. Through the evolutionary phase and progressive phase, PSTRN can converge to the interest region quickly and harvest good performance. Experimental results show that PSTRN can significantly reduce the complexity of seeking rank, compared with the enumerating method. Furthermore, our method is validated on public benchmarks like MNIST, CIFAR10/100, UCF11 and HMDB51, achieving the state-of-the-art performance.