JustSpeak: Automated, User-Configurable, Interactive Agents for Speech Tutoring
JustSpeak: Automated, User-Configurable, Interactive Agents for Speech Tutoring
复制标题
JustSpeak：用于语音辅导的自动化、用户可配置的交互式代理
DOI：
10.1145/3459744
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
and Jun Rekimoto
中科院分区：
文献类型：
--
作者：
Xinlei Zhang;Takashi Miyaki;and Jun Rekimoto
会话代理广泛应用于许多情况，尤其是语音辅导。然而，它们的内容和功能通常是预先定义的，并且对于没有技术背景的人来说是不可定制的，从而极大地限制了它们的灵活性和可用性。此外，传统代理通常无法在训练过程中提供反馈，因为它们缺乏动态评估用户语音的技术方法。我们提出 JustSpeak：具有各种可配置反馈机制的自动化和交互式语音辅导代理，使用任何语音录音及其转录文本作为语音训练的模板。在 JustSpeak 中，我们开发了一个自动化程序，可以根据用户输入的模板生成定制的辅导代理。此外，我们创建了一套方法来动态同步语音识别器的行为和代理的辅导进度，从而可以动态检测各种语音错误，例如卡住、发音错误和节奏偏差。此外，我们确定了 JustSpeak 中的设计原语，以创建不同的新颖反馈机制，例如自适应播放、后续训练和被动适应。它们可以组合起来创建定制的辅导代理，我们用语言学习的例子来演示这一点。我们相信 JustSpeak 可以通过支持可定制、始终可用且易于使用的辅导代理来创造更加个性化的语音学习机会。
Conversational agents are widely used in many situations, especially for speech tutoring. However, their contents and functions are often pre-defined and not customizable for people without technical backgrounds, thus significantly limiting their flexibility and usability. Besides, conventional agents often cannot provide feedback in the middle of training sessions because they lack technical approaches to evaluate users' speech dynamically. We propose JustSpeak: automated and interactive speech tutoring agents with various configurable feedback mechanisms, using any speech recordings with its transcription text as the template for speech training. In JustSpeak, we developed an automated procedure to generate customized tutoring agents from user-inputted templates. Moreover, we created a set of methods to dynamically synchronize speech recognizers' behavior with the agent's tutoring progress, making it possible to detect various speech mistakes dynamically such as being stuck, mispronunciation, and rhythm deviations. Furthermore, we identified the design primitives in JustSpeak to create different novel feedback mechanisms, such as adaptive playback, follow-on training, and passive adaptation. They can be combined to create customized tutoring agents, which we demonstrate with an example for language learning. We believe JustSpeak can create more personalized speech learning opportunities by enabling tutoring agents that are customizable, always available, and easy-to-use.