Non-lexical Sounds : a New Interface Modality for Voice-based Information Delivery Systems

本报告中收集的论文描述了在1999- 2000财政年度资助的题为“非词汇声音：基于语音的信息传递系统的新界面模式”的资助下进行的研究。在过去的5年里，交互式语音应答（IVR）系统在美国已经变得无处不在，并且正在日本取得进展。事实上，从一个真实的人那里获取火车时刻表、公寓信息、呼叫路由、航班信息、天气信息等等变得越来越不可能。这些系统被普遍讨厌的一个原因是需要听菜单，在菜单中导航，并按按钮选择内容，但由于语音识别技术的部署，这个困难正在得到解决。这些系统的第二个问题是，所提供的信息是以固定的块给出的，持续时间从几秒到几十秒，并且用户基本上被迫在系统回放块时收听。 关于我们 通过电话提供信息要灵活得多。其中一个方面是他们对来自听众的反馈做出反应。特别是，在许多对话类型中，听者经常会产生诸如uh-huh、uh、yeah-yeah、oh、ummmm等反向通道，而信息提供者则会相应地调整他的陈述。因此，本项目的目的是发现这些非词汇会话声音的意义和功能，该项目的第一个组成部分是对两种语言对话中非词汇声音的意义和功能进行基础研究，日语和英语。这导致了对这些声音的新理解。对于英语，这被形式化为一个模型，在这个模型中，这些项目被解释为不是固定的单词，而是动态的创造物，由一个简单的模型组成，该模型由10个组成声音和2个组合规则组成。该模型的语义成分是声音符号：每一个声音成分都具有某种意义或功能 * 在不同的语言和语境中是相当恒定的，而一个会话中的语言的意义很大程度上是它的语音成分的意义的总和。该项目的第二个组成部分是在一个教程系统中使用这些声音。该系统产生的非词汇的声音，适当地根据不同的上下文，展示了他们的效用在目标为导向的dialogs.The第三个组成部分的项目是一个研究的使用非词汇的声音在实时控制应用程序，其中计算机响应实时非词汇的意见，从用户，如mm-mm-mm或ack！会议议程上的其他议题包括：非词汇声音的韵律特征的意义的模型，2。日语会话中的非词汇声音模型，3.一种系统，其在指示辅助型IVR系统中的号码给出期间响应这些声音，以及4.一种基于这些声音调整其呈现的口语对话销售型系统。少

The papers collected in this report describe research performed under a grant entitled "Non-lexical Sounds : a New Interface Modality for Voice-based Information Delivery Systems" funded for fiscal 1999--2000.*er the past 5 years interactive voice response (IVR) systems have become ubiquitous in the United States and are making inroads in Japan. Indeed, it is becoming impossible to get train schedules, apartment information, call routing, flight information, weather information, and so on from a real person. One reason these systems are universally hated is the need to listen to menus, navigate through them, and push buttons to select content, but this difficulty is being resolved thanks to the deployment of speech recognition technology. The second problem with these systems is that the information provided is given in fixed chunks, lasting from a few seconds to a few tens of seconds, and the user is essentially forced to listen as the system plays back a chunk.In contrast, people pro … More viding information over the telephone are much more flexible. One aspect of this is that they are responsive to feedback from the listener. In particular, in many dialog types the listener frequently produces back-channels, such as uh-huh, uh, yeah-yeah, oh, ummmm and so on, and the information provider adapts his presentation in response.Thus the aim of this project was to discover the meanings and functions of such non-lexical conversational sounds, and to exploit them in voice-based information delivery systems.The first component of the project was basic research into the meanings and functions of non-lexical sounds in conversation in two languages, Japanese and English. This led to a new understanding of these sounds. For English, this was formalized as a model in which these items are explained, not as fixed words, but as dynamic creations, generated by a simple model consisting of 10 component sounds and 2 combining rules. The semantic component of the model is sound-symbolic : each of these component sounds bears some meaning or function * is fairly constant across grunts and across contexts, and the meaning of a conversational grunt is largely the sum of the meanings of its phonetic components.The second component of the project was an use of these sounds in a tutorial system. This system produced non-lexical sounds, suitably varied according to context, demonstrating their utility in goal-oriented dialogs.The third component of the project was a study of the use of non-lexical sounds in a real-time control application, in which the computer responded in real-time to non-lexical advice from the user, such as mm--mm--mm or ack! .Topics which remain on the agenda are : 1. a model of the meanings of the prosodic features of non-lexical sounds, 2. a model of non-lexical sounds in conversational Japanese, 3. a system which responds to these sounds during number-giving in a directory-assistance type IVR system, and 4. a spoken dialog salestalk-type system which adapts its presentation based on these sounds. Less