Attention-Based Speech Enhancement Using Human Quality Perception Modeling
Attention-Based Speech Enhancement Using Human Quality Perception Modeling
复制标题
DOI：
10.1109/taslp.2023.3328282
复制
发表时间：
2023-03
期刊：
影响因子：
--
通讯作者：
Khandokar Md. Nayem;D. Williamson
中科院分区：
文献类型：
--
作者：
Khandokar Md. Nayem;D. Williamson
感知启发的目标函数，如语音质量的感知评价（PESQ）、信号失真比（SDR）和短时目标可理解性（STOI），最近被用于优化基于深度学习的语音增强算法的性能。然而，这些目标函数并不总是与听众对感知质量的评估密切相关，因此，在现实场景中，使用这些措施进行优化通常会导致较差的表现。在这项工作中，我们提出了一种基于注意力的增强方法，该方法使用来自平均意见评分（MOS）预测模型的学习语音嵌入向量和语音增强模块来共同增强带噪语音。MOS预测模型直接从音频信号中估计由人类听众评估的语音质量的感知MOS。增强模块还采用了量化语言模型，该模型强制执行频谱约束，以获得更好的语音真实感和性能。我们使用在日常环境中捕获的真实噪声语音数据来训练模型，并使用未见过的语料库进行测试。结果表明，我们提出的方法明显优于其他使用客观度量进行优化的方法，其中预测的质量分数与人类判断密切相关。
Perceptually-inspired objective functions such as the perceptual evaluation of speech quality (PESQ), signal-to-distortion ratio (SDR), and short-time objective intelligibility (STOI), have recently been used to optimize performance of deep-learning-based speech enhancement algorithms. These objective functions, however, do not always strongly correlate with a listener's assessment of perceptual quality, so optimizing with these measures often results in poorer performance in real-world scenarios. In this work, we propose an attention-based enhancement approach that uses learned speech embedding vectors from a mean-opinion score (MOS) prediction model and a speech enhancement module to jointly enhance noisy speech. The MOS prediction model estimates the perceptual MOS of speech quality, as assessed by human listeners, directly from the audio signal. The enhancement module also employs a quantized language model that enforces spectral constraints for better speech realism and performance. We train the model using real-world noisy speech data that has been captured in everyday environments and test it using unseen corpora. The results show that our proposed approach significantly outperforms other approaches that are optimized with objective measures, where the predicted quality scores strongly correlate with human judgments.