Automatic Transcription of Conversations

多人对话语音识别涉及使用远程麦克风以机器可读形式转录正式会议或非正式聚会的音频记录。目前的解决方案远远没有达到人类的表现。这项任务的难度可归因于三个因素。首先，录音条件具有挑战性：麦克风从远处捕获的语音信号具有噪声和混响，并且通常包含非平稳的声学失真，这使得难以解码。第二，有很大比例的时间与重叠的语音，其中多个扬声器在同一时间说话。最后，该场景的交互动态是具有挑战性的，因为说话者以间歇性的方式表达自己，其中语音不活动、单说话者语音和多说话者语音的交替片段。我们的目标是开发一个转录系统，能够操作任意长度的输入，正确地处理段重叠以及非重叠的语音，并转录不同的扬声器的语音始终到单独的输出流。虽然使用单独训练的子系统进行日志化，分离和识别的现有方法到目前为止还不能达到人类的性能，但我们认为缺少的部分是一种配方，它封装了会议转录的所有方面，并且允许在单一优化标准下设计联合方法。该项目旨在制定这样一个连贯的方案。我们将开发一个集成的解决方案，用于使用远距离麦克风的多说话者会话语音识别，其中一次活动的说话者数量和说话者重叠的程度是未知的且随时间变化的。将在一个共同的目标函数下处理日记化、增强和识别的任务，最终接近端到端的培训。作为补充，我们打算以端到端的识别为目标，避免过早做出决定。我们将推导和考虑不同的架构（全神经，混合，级联，集成），并评估它们的w.r.t.日记化和转录性能，以及w.r.t.中间结果的可解释性和相应配方的易用性。

Multi-talker conversational speech recognition is concerned with the transcription of audio recordings of formal meetings or informal get-togethers in machine-readable form using distant microphones. Current solutions are far from reaching human performance. The difficulty of the task can be attributed to three factors. First, the recording conditions are challenging: The speech signal captured by microphones from a distance is noisy and reverberated and often contains nonstationary acoustic distortions, which makes it hard to decode. Second, there is a significant percentage of time with overlapped speech, where multiple speakers talk at the same time. Finally, the interaction dynamics of the scenario are challenging because speakers articulate themselves in an intermittent manner with alternating segments of speech inactivity, single-, and multi-talker speech. We aim to develop a transcription system that is able to operate on arbitrary length input, correctly handles segments of overlapped as well as non-overlapped speech, and transcribes the speech of different speakers consistently into separate output streams. While existing approaches using separately trained subsystems for diarization, separation, and recognition are by far not able to reach human performance, we believe that the missing piece is a formulation which encapsulates all aspects of meeting transcription and which allows to design a joint approach under a single optimization criterion. This project is aimed at such a coherent formulation. We are going to develop an integrated solution for multi-talker conversational speech recognition using distant microphones, where the number of speakers who are active at a time and the extent of speaker overlap is unknown and time-varying. The tasks of diarization, enhancement and recognition will be addressed under a common objective function, ultimately approaching end-to-end training. Complementarily, we intend to avoid premature decisions with the goal of end-to-end recognition. We will derive and consider different architectures (all-neural, hybrid, cascaded, integrated), and assess them w.r.t. diarization and transcription performance, as well as w.r.t. the interpretability of intermediate results and the ease of use of the corresponding recipes.