Challenges in CISE: Creating Conversational Agents for Language Training: Technologies for the Next Generation of Interactive Systems

CISE 中的挑战：创建用于语言训练的对话代理：下一代交互式系统的技术

• 批准号: 9726363
• 负责人: Ronald Cole
• 金额: $ 121.09万
• 依托单位: Oregon Graduate Institute of Science & Technology
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 1997
• 资助国家: 美国
• 起止时间: 1997-09-15 至 1999-04-22
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=9726363&HistoricalAwards=false
• 关键词: Challenges; CISE; Creating; Conversational; Agents

9726363 Cole, Ronald Massaro, Dominic Oregon Graduate Institute of Science Challenges in CISE: Creating Conversational Agents for Language Training Technologies for the Next Generation of Interactive Systems The challenge addressed by this research is the creation of a realistic conversational agent for the domain of language training, and for spoken language training of the hearing impaired in particular. The conversational agent consists of four key technologies, two for output and two for input: (1) a 3D model of a talking face with accurate movement of the articulators (lips, tongue, etc.) and a variety of facial gestures and expression, (2) natural text-to-speech based on concatenative synthesis which can quickly learn new voices and is capable of hyperarticulation of the kind used to provide feedback in language training, (3) visual speech recognition using an unobtrusive desk-top camera to aid speech recognition and to provide information about the articulators of the student, and (4) auditory speech recognition so the system can understand what is said and also so the system can detect mispronunciations. Research advances are required in speech recognition and understanding, language generation and speech synthesis, recognition of facial cues and head movements, and generation of speech by artificial talking heads to improve the accuracy and to tailor the technology to conversational agents in general and language training in particular. Participatory design experiments will be conducted with schools in the Portland area. The Tucker-Maxon Oral School will use the conversational agents to teach hearing-impaired students to use and understanding auditory and visual speech. This realistic testbed will provide the research team with invaluable feedback on the usefulness of the system.

9726363科尔，罗纳德马萨罗，多米尼克俄勒冈州科学研究生院 CISE的挑战：为下一代交互系统的语言训练技术创建会话代理 这项研究所面临的挑战是创建一个现实的会话代理的语言训练领域，特别是听力障碍者的口语训练。 会话代理包括四个关键技术，两个用于输出，两个用于输入：（1）具有发音器官（嘴唇，舌头等）精确运动的说话面部的3D模型。以及各种面部手势和表情，（2）基于拼接合成的自然文本到语音转换，可以快速学习新的声音，并且能够进行用于在语言训练中提供反馈的那种超发音，（3）视觉语音识别使用不引人注目的桌面摄像头来帮助语音识别并提供有关学生发音器官的信息，以及（4）听觉语音识别，使得系统能够理解所说的内容，并且还使得系统能够检测错误发音。 需要在语音识别和理解，语言生成和语音合成，面部线索和头部运动的识别，以及人工说话头的语音生成方面取得研究进展，以提高准确性，并将技术定制为一般的会话代理和特别是语言训练。 将在波特兰地区的学校进行先行设计实验。 塔克马克森口腔学校将使用会话代理教听力受损的学生使用和理解听觉和视觉语音。 这个现实的测试平台将为研究团队提供关于系统有用性的宝贵反馈。