TRADER: Trace Divergence Analysis and Embedding Regulation for Debugging Recurrent Neural Networks
TRADER: Trace Divergence Analysis and Embedding Regulation for Debugging Recurrent Neural Networks
复制标题
DOI：
10.1145/3377811.3380423
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Guanhong Tao;Shiqing Ma;Yingqi Liu;Qiuling Xu;X. Zhang
中科院分区：
文献类型：
--
作者：
Guanhong Tao;Shiqing Ma;Yingqi Liu;Qiuling Xu;X. Zhang
递归神经网络（RNN）可以处理各种长度的文本输入，因此在软件系统和软件工程应用中有着广泛的应用。RNN依赖于通常由第三方预先训练的单词嵌入，以将文本输入编码为数值。众所周知，有问题的词嵌入会导致模型准确度低。在本文中，我们提出了一种新的技术来自动诊断有问题的嵌入如何影响模型的性能，通过比较正确和不正确执行的样本的模型执行轨迹。然后，我们利用诊断结果作为指导来硬化/修复嵌入。我们的实验表明，TRADER可以持续有效地提高真实的世界模型和数据集的准确性，平均提高5.37%，这代表了RNN模型文献的实质性改进。
Recurrent Neural Networks (RNN) can deal with (textual) input with various length and hence have a lot of applications in software systems and software engineering applications. RNNs depend on word embeddings that are usually pre-trained by third parties to encode textual inputs to numerical values. It is well known that problematic word embeddings can lead to low model accuracy. In this paper, we propose a new technique to automatically diagnose how problematic embeddings impact model performance, by comparing model execution traces from correctly and incorrectly executed samples. We then leverage the diagnosis results as guidance to harden/repair the embeddings. Our experiments show that TRADER can consistently and effectively improve accuracy for real world models and datasets by 5.37% on average, which represents substantial improvement in the literature of RNN models.