Self-supervised graph-based representation for language and speaker detection

我专注于研究如何更好地表示语言识别和语音识别任务的语音信号。具体来说，为推进本项目，主要做了以下工作：1.项目进度；改进语言识别（LID）语音信号的表示：我们提出了一种新的基于换能器的语言嵌入方法，通过将RNN换能器模型集成到语言嵌入框架中。利用RNN换能器语言表征能力的优势，该方法可以同时利用语音感知声学特征和显式语言特征来完成LID任务。该研究论文被Interspeech 2022收录。此外，我们在NICT LID系统上进一步研究了这些技术，也证明了跨通道数据的鲁棒性。另一项工作着重于改进汉语ASR的RNN-T。我建议使用一种新颖的发音感知独特的字符编码来构建端到端的基于rnn的普通话ASR系统。提出的编码是基于发音的音节和字符索引（CI）的组合。通过引入CI， RNN-T模型在利用语音信息提取建模单元的同时克服了同音问题。利用所提出的编码方法，可以通过一对一映射将模型输出转换为最终识别结果。论文已被IEEE SLT 2022接受。

I focused on investigating how to better represent speech signals for both language recognition and speech recognition tasks. In detail, the following work was done to progress this project:1. Improving the representation of speech signal for language identification (LID): We propose a novel transducer-based language embedding approach for LID tasks by integrating an RNN transducer model into a language embedding framework. Benefiting from the advantages of the RNN transducer's linguistic representation capability, the proposed method can exploit both phonetically-aware acoustic features and explicit linguistic features for LID tasks. The research paper was accepted by Interspeech 2022. Additionally, we further investigated these techniques on the NICT LID system, which also demonstrated robustness on cross-channel data.2. Another work focuses on improving RNN-T for Mandarin ASR. I propose to use a novel pronunciation-aware unique character encoding for building end-to-end RNN-T-based Mandarin ASR systems. The proposed encoding is a combination of pronunciation-based syllable and character index (CI). By introducing the CI, the RNN-T model can overcome the homophone problem while utilizing the pronunciation information for extracting modeling units. With the proposed encoding, the model outputs can be converted into the final recognition result through a one-to-one mapping. This paper was accepted by IEEE SLT 2022.