F-Measure Based End-to-End Optimization of Neural Network Keyword Detectors
F-Measure Based End-to-End Optimization of Neural Network Keyword Detectors
复制标题
DOI：
10.23919/apsipa.2018.8659736
复制
发表时间：
2018-11
期刊：
影响因子：
--
通讯作者：
Tomohiro Tanaka;T. Shinozaki
中科院分区：
文献类型：
--
作者：
Tomohiro Tanaka;T. Shinozaki
F-measure是一种广泛使用的关键词检测评估标准，其中准确度和交叉熵等指标并不合适，因为阳性和阴性样本的数量在很大程度上是不平衡的。在这项工作中，我们提出了一个软决策版本的F-测量作为目标函数来训练端到端的神经网络基于关键字检测器。其优点是培训目标与最终评价指标一致。我们将其应用于使用长短期记忆（LSTM）的基于声学嵌入的关键字检测器。使用WSJ语料库的评估实验表明，所提出的基于F度量的训练比交叉熵训练提高了关键字检测性能，同时消除了一个元参数，以平衡训练中正负样本的影响。此外，我们提出了端到端的连续动态规划（DP）匹配使用二维递归神经网络（2D-RNN），并训练它与建议的F-测量标准。虽然它在训练中具有不稳定的行为，但比嵌入方法获得了更高的检测性能。
F-measure is a widely used evaluation criterion for keyword detection where measures such as accuracy and cross-entropy are not appropriate since the numbers of positive and negative samples are largely unbalanced. In this work, we propose a soft decision version of the F-measure as an objective function to train end-to-end neural network based keyword detectors. It has advantages that the training objective is consistent with the final evaluation measure. We apply it to acoustic embedding based keyword detectors using long short term memory (LSTM). Evaluation experiments using the WSJ corpus show that the proposed F-measure based training improves keyword detection performance than cross-entropy training while eliminating a meta-parameter to balance the effect of positive and negative samples in training. Additionally, we propose end-to-end continuous dynamic programming (DP) matching using a two-dimensional recurrent neural network (2D-RNN) and train it with the proposed F-measure criterion. While it has unstable behavior in the training, higher detection performance than the embedding method has achieved.