An Efficient Video Prediction Recurrent Network using Focal Loss and Decomposed Tensor Train for Imbalance Dataset
An Efficient Video Prediction Recurrent Network using Focal Loss and Decomposed Tensor Train for Imbalance Dataset
复制标题
DOI：
10.1145/3453688.3461748
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
Mingshuo Liu;Kevin Han;Shiying Luo;Mingze Pan;M. Hossain;Bo Yuan;R. Demara;Y. Bai
中科院分区：
文献类型：
--
作者：
Mingshuo Liu;Kevin Han;Shiying Luo;Mingze Pan;M. Hossain;Bo Yuan;R. Demara;Y. Bai
如今，从公司到学术界，世界各地的研究人员都对开发递归神经网络感兴趣，因为它们在语音识别、视频检测、预测和机器翻译等各种应用中具有令人难以置信的功能。然而，递归神经网络的优点伴随着高计算和功率需求，这对于在这种网络实现中使用的具有有限资源的电子设备是主要的设计约束。优化递归神经网络（如模型压缩）对于确保递归神经网络的广泛部署和促进递归神经网络实现大多数资源受限的场景至关重要。在众多技术中，张量序列（TT）分解被认为是一种新兴的技术。虽然我们以前的努力已经实现了1）在消除所有冗余计算的范围内扩展许多乘法的限制; 2）分解为多阶段处理以减少内存流量，但这项工作仍然面临一些限制。特别是，当前递归神经网络上的TT分解导致对训练数据集质量敏感的复杂计算。在本文中，我们研究了一种新的方法TT分解递归神经网络构建一个有效的模型不平衡的数据集，以克服这个问题。实验结果表明，新的训练方法在准确率、精确率、召回率、F1分数、假阴性率（FNR）和假遗漏率（FOR）等方面都有显著的提高。
Nowadays, from companies to academics, researchers across the world are interested in developing recurrent neural networks due to their incredible feats in various applications, such as speech recognition, video detection, predictions, and machine translation. However, the advantages of recurrent neural networks accompanied by high computational and power demands, which are a major design constraint for electronic devices with limited resources used in such network implementations. Optimizing the recurrent neural networks, such as model compression, is crucial to ensure the broad deployment of recurrent neural networks and promote recurrent neural networks for implementing most resource-constrained scenarios. Among many techniques, tensor train (TT) decomposition is considered an up-and-coming technology. Although our previous efforts have achieved 1) expanding limits of many multiplications within eliminating all redundant computations; and 2) decomposing into multi-stage processing to reduce memory traffic, this work still faces some limitations. In particular, current TT decomposition on recurrent neural networks leads to a complex computation sensitive to the quality of training datasets. In this paper, we investigate a new method for TT decomposition on recurrent neural networks for constructing an efficient model within imbalance datasets to overcome this issue. Experimental results show that the proposed new training method can achieve significant improvements in accuracy, precision, recall, F1-score, False Negative Rate (FNR), and False Omission Rate (FOR).