Using Deep Time Delay Neural Network for Slot Filling in Spoken Language Understanding
Using Deep Time Delay Neural Network for Slot Filling in Spoken Language Understanding
复制标题
使用深度时延神经网络进行口语理解中的槽填充
DOI：
10.3390/sym12060993
复制
发表时间：
2020
期刊：
影响因子：
2.7
通讯作者：
Wang Kai
中科院分区：
文献类型：
--
作者：
Zhang Zhen;Huang Hao;Wang Kai
建立目标词的上下文模型对于口语理解中的槽填充任务的语义标签预测具有重要意义。虽然递归神经网络（RNN）已经成功地实现了SLU的最先进的结果，并且双向RNN能够通过不仅从过去而且从未来建模信息来获得进一步的改进，但它们只考虑了目标词的有限上下文信息。为了使网络更深，从而获得更长的上下文信息，我们建议使用多层时间延迟神经网络（TDNN），这是目前大词汇量连续语音识别任务中流行的。特别地，我们使用具有对称时间延迟偏移的TDNN。为了使堆叠式TDNN易于训练，采用了残差结构和跳跃连接。此外，我们通过引入ResTDNN-BiLSTM进一步改进了模型，它结合了残余TDNN和BiLSTM的优点。在航空旅行信息系统（ATIS）和Snips基准数据集上进行的时隙填充任务实验表明，所提出的SC-TDNN-C在没有任何额外知识和数据资源的情况下实现了最先进的结果。最后，我们回顾和比较槽填充的结果，通过使用各种现有的模型和方法。
Modeling the context of a target word is of fundamental importance in predicting the semantic label for slot filling task in Spoken Language Understanding (SLU). Although Recurrent Neural Network (RNN) has shown to successfully achieve the state-of-the-art results for SLU, and Bidirectional RNN is capable of obtaining further improvement by modeling information not only from the past, but also from the future, they only consider limited contextual information of the target word. In order to make the network deeper and hence obtain longer contextual information, we propose to use a multi-layer Time Delay Neural Network (TDNN), which is prevalent in current large vocabulary continuous speech recognition tasks. In particular, we use a TDNN with symmetric time delay offset. To make the stacked TDNN easily trained, residual structures and skip concatenation are adopted. In addition, we further improve the model by introducing ResTDNN-BiLSTM, which combines the advantages of both the residual TDNN and BiLSTM. Experiments on slot filling tasks on the Air Travel Information System (ATIS) and Snips benchmark datasets show the proposed SC-TDNN-C achieves state-of-the-art results without any additional knowledge and data resources. Finally, we review and compare slot filling results by using a variety of existing models and methods.