Non-lexical Sounds : a New Interface Modality for Voice-based Information Delivery Systems

非词汇声音:基于语音的信息传递系统的新接口模式

• 批准号: 11680412
• 负责人: WARD Nigel
• 金额: $ 1.6万
• 依托单位: University Of Tokyo
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (C)
• 财政年份: 1999
• 资助国家: 日本
• 起止时间: 1999 至 2000
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-11680412/
• 关键词: modality; non-verbal communication; second channnel; utterance; phonetic components; conversation analysis; user interface; spoken dialog system; 非言語情報; 対話系インタフェース; 情報提供; 音響分析; barge-in

The papers collected in this report describe research performed under a grant entitled "Non-lexical Sounds : a New Interface Modality for Voice-based Information Delivery Systems" funded for fiscal 1999--2000.*er the past 5 years interactive voice response (IVR) systems have become ubiquitous in the United States and are making inroads in Japan. Indeed, it is becoming impossible to get train schedules, apartment information, call routing, flight information, weather information, and so on from a real person. One reason these systems are universally hated is the need to listen to menus, navigate through them, and push buttons to select content, but this difficulty is being resolved thanks to the deployment of speech recognition technology. The second problem with these systems is that the information provided is given in fixed chunks, lasting from a few seconds to a few tens of seconds, and the user is essentially forced to listen as the system plays back a chunk.In contrast, people pro … More viding information over the telephone are much more flexible. One aspect of this is that they are responsive to feedback from the listener. In particular, in many dialog types the listener frequently produces back-channels, such as uh-huh, uh, yeah-yeah, oh, ummmm and so on, and the information provider adapts his presentation in response.Thus the aim of this project was to discover the meanings and functions of such non-lexical conversational sounds, and to exploit them in voice-based information delivery systems.The first component of the project was basic research into the meanings and functions of non-lexical sounds in conversation in two languages, Japanese and English. This led to a new understanding of these sounds. For English, this was formalized as a model in which these items are explained, not as fixed words, but as dynamic creations, generated by a simple model consisting of 10 component sounds and 2 combining rules. The semantic component of the model is sound-symbolic : each of these component sounds bears some meaning or function * is fairly constant across grunts and across contexts, and the meaning of a conversational grunt is largely the sum of the meanings of its phonetic components.The second component of the project was an use of these sounds in a tutorial system. This system produced non-lexical sounds, suitably varied according to context, demonstrating their utility in goal-oriented dialogs.The third component of the project was a study of the use of non-lexical sounds in a real-time control application, in which the computer responded in real-time to non-lexical advice from the user, such as mm--mm--mm or ack! .Topics which remain on the agenda are : 1. a model of the meanings of the prosodic features of non-lexical sounds, 2. a model of non-lexical sounds in conversational Japanese, 3. a system which responds to these sounds during number-giving in a directory-assistance type IVR system, and 4. a spoken dialog salestalk-type system which adapts its presentation based on these sounds. Less

本报告中收集的论文描述了在1999- 2000财政年度资助的题为“非词汇声音：基于语音的信息传递系统的新接口模式”的资助下进行的研究。在过去的5年里，交互式语音应答（IVR）系统在美国变得无处不在，并且正在进军日本。事实上，从真人那里获得火车时刻表、公寓信息、电话路由、航班信息、天气信息等信息已经变得不可能了。人们普遍讨厌这些系统的一个原因是，它们需要听菜单，在菜单中导航，并按下按钮来选择内容，但由于语音识别技术的部署，这个困难正在得到解决。这些系统的第二个问题是，所提供的信息是以固定块的形式提供的，持续时间从几秒到几十秒不等，当系统回放一个块时，用户实际上是被迫倾听的。相比之下，人们倾向于通过电话提供更多的视频信息，这要灵活得多。其中一个方面是，他们会对听众的反馈做出反应。特别是，在许多对话类型中，听者经常产生反向通道，例如uh-huh， uh, yeah-yeah, oh， ummmm等等，而信息提供者则调整他的表达作为回应。因此，这个项目的目的是发现这些非词汇会话声音的意义和功能，并在基于语音的信息传递系统中加以利用。该项目的第一个组成部分是对日语和英语两种语言对话中非词性声音的意义和功能进行基础研究。这让我们对这些声音有了新的理解。对于英语来说，这被形式化为一个模型，在这个模型中，这些项目不是固定的单词，而是动态的创造，由一个由10个组成音和2个组合规则组成的简单模型产生。该模型的语义组成部分是声音符号：这些组成部分中的每一个声音都具有一定的意义或功能*在咕噜声和上下文中是相当恒定的，会话咕噜声的意义在很大程度上是其语音组成部分意义的总和。项目的第二个组成部分是在教程系统中使用这些声音。这个系统产生非词汇的声音，根据上下文适当地变化，显示出它们在目标导向对话中的效用。该项目的第三个组成部分是研究在实时控制应用程序中使用非词汇声音，其中计算机实时响应来自用户的非词汇建议，例如mm- mm- mm或ack！。仍留在议程上的议题有：1。非词汇语音韵律特征的意义模型，2。2 .日语会话中的非词性发音模式；3 .在目录辅助型IVR系统中，在给号码时响应这些声音的系统；一种基于这些声音的口语对话销售系统。少

项目成果

Nigel Ward: "Sound Symbolism in Conversational Grunts in English (submitted)"Language and Speech. (submitted). (2001)

奈杰尔·沃德（Nigel Ward）：“英语会话咕噜声中的声音象征（已提交）”语言和言语。

Wataru Tsukahara and Nigel Ward: "Responding to Subtle, Fleeting Changes in the User's Internal State"CHI 2001 Conference on Human Factors in Computing Systems. (to appear).

Wataru Tsukahara 和 Nigel Ward：“响应用户内部状态中微妙、短暂的变化”CHI 2001 年计算系统人为因素会议。

Nigel Ward: "The Challenge of Non-lexical Speech Sounds"International Conference on Spoken Language Processing, 2000. 571-574 (2000)

Nigel Ward：“非词汇语音的挑战”国际口语处理会议，2000. 571-574 (2000)

Wataru Tsukahara and Nigel Ward: "Evaluating Responsiveness in Spoken Dialog Systems"International Conference of Spoken Language Processing 2000. 1097-1100 (2000)

Wataru Tsukahara 和 Nigel Ward：“评估口语对话系统中的响应性”2000 年口语处理国际会议。1097-1100 (2000)

Shunsuke Soeda and Nigel Ward: "Hard-Realtime Communication between Human and Computer using Non-Verbal Information."Interaction 2001 (87th Human Interface Workshop), Information Processing Society of Japan 2001. 47-48 (2001)

Shunsuke Soeda 和 Nigel Ward：“使用非语言信息在人与计算机之间进行硬实时通信。”Interaction 2001（第 87 届人机界面研讨会），日本信息处理协会 2001. 47-48 (2001)