Challenges in CISE: Creating Conversational Agents for Language Training: Technologies for the Next Generation of Interactive Systems

9726363科尔，罗纳德马萨罗，多米尼克俄勒冈州科学研究生院 CISE的挑战：为下一代交互系统的语言训练技术创建会话代理 这项研究所面临的挑战是创建一个现实的会话代理的语言训练领域，特别是听力障碍者的口语训练。 会话代理包括四个关键技术，两个用于输出，两个用于输入：（1）具有发音器官（嘴唇，舌头等）精确运动的说话面部的3D模型。以及各种面部姿势和表情，（2）基于连接合成的自然文本到语音，其可以快速学习新的语音并且能够进行用于在语言训练中提供反馈的那种超发音，（3）使用不显眼的台式摄像机来辅助语音识别并且提供关于学生发音器官的信息的视觉语音识别，以及（4）听觉语音识别，使得系统能够理解所说的内容，并且还使得系统能够检测错误发音。 需要在语音识别和理解，语言生成和语音合成，面部线索和头部运动的识别，以及人工说话头的语音生成方面取得研究进展，以提高准确性，并将技术定制为一般的会话代理和特别是语言训练。 将在波特兰地区的学校进行先行设计实验。 塔克马克森口腔学校将使用会话代理教听力受损的学生使用和理解听觉和视觉语音。 这个现实的测试平台将为研究团队提供关于系统有用性的宝贵反馈。

9726363 Cole, Ronald Massaro, Dominic Oregon Graduate Institute of Science Challenges in CISE: Creating Conversational Agents for Language Training Technologies for the Next Generation of Interactive Systems The challenge addressed by this research is the creation of a realistic conversational agent for the domain of language training, and for spoken language training of the hearing impaired in particular. The conversational agent consists of four key technologies, two for output and two for input: (1) a 3D model of a talking face with accurate movement of the articulators (lips, tongue, etc.) and a variety of facial gestures and expression, (2) natural text-to-speech based on concatenative synthesis which can quickly learn new voices and is capable of hyperarticulation of the kind used to provide feedback in language training, (3) visual speech recognition using an unobtrusive desk-top camera to aid speech recognition and to provide information about the articulators of the student, and (4) auditory speech recognition so the system can understand what is said and also so the system can detect mispronunciations. Research advances are required in speech recognition and understanding, language generation and speech synthesis, recognition of facial cues and head movements, and generation of speech by artificial talking heads to improve the accuracy and to tailor the technology to conversational agents in general and language training in particular. Participatory design experiments will be conducted with schools in the Portland area. The Tucker-Maxon Oral School will use the conversational agents to teach hearing-impaired students to use and understanding auditory and visual speech. This realistic testbed will provide the research team with invaluable feedback on the usefulness of the system.