Learning to Adaptively Scale Recurrent Neural Networks
Learning to Adaptively Scale Recurrent Neural Networks
复制标题
DOI：
10.1609/aaai.v33i01.33013822
复制
发表时间：
2019-02
期刊：
影响因子：
--
通讯作者：
Hao Hu;Liqiang Wang;Guo-Jun Qi
中科院分区：
文献类型：
--
作者：
Hao Hu;Liqiang Wang;Guo-Jun Qi
递归神经网络（RNN）研究的最新进展已经证明了利用多尺度结构学习时间序列的时间表示的优越性。目前，大多数多尺度RNN使用固定的尺度，这不符合序列之间的动态时间模式的性质。在本文中，我们提出了自适应缩放递归神经网络（ASRNN），这是一种简单但有效的方法来处理这个问题。ASRNN不使用预定义的尺度，而是能够根据不同的时间上下文学习和调整尺度，使它们在建模多尺度模式时更加灵活。与其他多尺度RNN相比，ASRNN具有动态缩放能力，结构简单，易于与各种RNN单元集成。多个序列建模任务的实验表明，ASRNN可以有效地适应不同的序列上下文的尺度，并产生更好的性能比基线没有动态缩放能力。
Recent advancements in recurrent neural network (RNN) research have demonstrated the superiority of utilizing multiscale structures in learning temporal representations of time series. Currently, most of multiscale RNNs use fixed scales, which do not comply with the nature of dynamical temporal patterns among sequences. In this paper, we propose Adaptively Scaled Recurrent Neural Networks (ASRNN), a simple but efficient way to handle this problem. Instead of using predefined scales, ASRNNs are able to learn and adjust scales based on different temporal contexts, making them more flexible in modeling multiscale patterns. Compared with other multiscale RNNs, ASRNNs are bestowed upon dynamical scaling capabilities with much simpler structures, and are easy to be integrated with various RNN cells. The experiments on multiple sequence modeling tasks indicate ASRNNs can efficiently adapt scales based on different sequence contexts and yield better performances than baselines without dynamical scaling abilities.