Training deep neural-networks using a noise adaptation layer
Training deep neural-networks using a noise adaptation layer
复制标题
DOI：
--
复制
发表时间：
2016-11
期刊：
影响因子：
--
通讯作者：
J. Goldberger;Ehud Ben-Reuven
中科院分区：
文献类型：
--
作者：
J. Goldberger;Ehud Ben-Reuven
大型数据集的可用性使神经网络能够取得令人印象深刻的识别结果。然而，众所周知，不准确的类标签的存在会降低即使是最好的分类器在广泛的分类问题中的性能。嘈杂的标签往往比嘈杂的属性更有害。当观察到的标签有噪声时，我们可以将正确的标签视为潜在的随机变量，并通过具有未知参数的通信通道对噪声过程进行建模。因此我们可以应用 EM 算法来找到网络和噪声的参数并估计正确的标签。在本研究中，我们提出了一种神经网络方法，该方法优化与 EM 算法优化相同的似然函数。噪声由附加的 softmax 层显式建模，该层将正确的标签与噪声标签连接起来。然后将该方案扩展到噪声标签除了正确标签之外还依赖于特征的情况。实验结果表明该方法优于以前的方法。
The availability of large datsets has enabled neural networks to achieve impressive recognition results. However, the presence of inaccurate class labels is known to deteriorate the performance of even the best classifiers in a broad range of classi-fication problems. Noisy labels also tend to be more harmful than noisy attributes. When the observed label is noisy, we can view the correct label as a latent random variable and model the noise processes by a communication channel with unknown parameters. Thus we can apply the EM algorithm to find the parameters of both the network and the noise and estimate the correct label. In this study we present a neural-network approach that optimizes the same likelihood function as optimized by the EM algorithm. The noise is explicitly modeled by an additional softmax layer that connects the correct labels to the noisy ones. This scheme is then extended to the case where the noisy labels are dependent on the features in addition to the correct labels. Experimental results demonstrate that this approach outperforms previous methods.