The Potential of Neural Speech Synthesis-Based Data Augmentation for Personalized Speech Enhancement
The Potential of Neural Speech Synthesis-Based Data Augmentation for Personalized Speech Enhancement
复制标题
DOI：
10.1109/icassp49357.2023.10096601
复制
发表时间：
2022-11
期刊：
影响因子：
--
通讯作者：
Anastasia Kuznetsova;Aswin Sivaraman;Minje Kim
中科院分区：
文献类型：
--
作者：
Anastasia Kuznetsova;Aswin Sivaraman;Minje Kim
随着深度学习的进步，语音增强系统受益于大型神经网络架构并实现了最先进的质量。然而，当在资源受限的环境中使用与说话人无关的方法时，无论是在质量还是复杂性方面，它们并不总是令人满意的。一种有前途的方法是个性化语音增强（PSE），这是一种更小、更容易解决的语音增强问题，适合小模型解决，因为它专注于特定的测试时用户。为了实现个性化目标，在处理典型的个人数据缺乏的同时，我们研究了基于神经语音合成（NSS）的数据增强的效果。在所提出的方法中，我们表明 NSS 系统的合成数据的质量很重要，如果它们足够好，则可以使用增强数据集来改进 PSE 系统，使其性能优于与说话人无关的基线。所提出的 PSE 系统在保持增强质量的同时显着降低了复杂性。
With the advances in deep learning, speech enhancement systems benefited from large neural network architectures and achieved state-of-the-art quality. However, speaker-agnostic methods are not always desirable, both in terms of quality and their complexity, when they are to be used in a resource-constrained environment. One promising way is personalized speech enhancement (PSE), which is a smaller and easier speech enhancement problem for small models to solve, because it focuses on a particular test-time user. To achieve the personalization goal, while dealing with the typical lack of personal data, we investigate the effect of data augmentation based on neural speech synthesis (NSS). In the proposed method, we show that the quality of the NSS system’s synthetic data matters, and if they are good enough the augmented dataset can be used to improve the PSE system that outperforms the speaker-agnostic baseline. The proposed PSE systems show significant complexity reduction while preserving the enhancement quality.