CAREER: Enhancing Vocal Communication using Graphical Social Cues

该提案概述了一个具有挑战性的目标，即将语音的易用性与图形的视觉反馈相结合，以创建一种新的通信媒介。该项目将创建可视化通信的图形语言。图形语言的不同版本将用于调解远程和同地对话，创建学习工具以获得新的语言技能和进行语言治疗，创建结合时间和相位分析的新的可视化技术，并产生存档音频，语音和声音的新方法。知识价值。这个项目的智力优势在于从不同于传统音频工程师的角度和目标来考虑音频、语音和语音处理。该项目从尚未研究的简单而有效的技术开始，并发展到开发新的分析和可视化技术。图形语言将通过适合其使用的应用程序进行评估。本研究研究了在中介互动中比面对面互动中更明显的社会线索和信号的使用；声音中最有效、最容易解释的参数；使用颜色，形状和运动映射到这些声音参数。这项研究将帮助我们理解，在同一地点和远程空间中，公开的可视化对话如何影响我们的声音互动；以及将语音数据与图形数据结合起来能增加多少价值。更广泛的影响。这项研究将改变在线环境和物理空间中的声音交流互动。实时存档的行为将对语音通信和协作产生新的影响。这项研究将为帮助用户学习发声的微妙之处提供新的工具。这些工具将用于学习新语言和语言治疗。该研究将发声工具扩展到有形的玩具对象，以鼓励有社会障碍的儿童发声。这项研究将提供一种替代的图形方法来存档大量音频，这些音频可以一眼搜索到，而不是通过搜索引擎。研究结果将在新课程和研究出版物中传播。

This proposal outlines a challenging goal of combining the ease of voice with the visual feedback of graphics to create a new communication medium. The project will create a graphical language for visualizing communication. Different versions of the graphical language will be used to mediate remote and co-located conversation, to create learning tools for acquiring new language skills and conducting speech therapy, to create new visualization techniques combining time and phase analysis, and to produce novel methods of archiving audio, speech, and voice.Intellectual Merit. The intellectual merit of this project is the consideration audio, speech, and voice processing from a perspective and goal different from that of traditional audio engineers. The project begins with simple yet effective techniques that have not yet been studied and progresses to developing new analysis and visualization techniques. The graphical languages will be evaluated through applications catered to their use. This research studies the use of social cues and signals that are more readily apparent in mediated interaction than in face-to-face interaction; the parameters in voice that are most effective and easy to interpret; the use of color, shape, and motion mapped onto these voice parameters. The research will help us understand how visualizing conversation publicly affects our vocal interaction when in co-located spaces and in remote spaces; and how much value is added by combining voice data with graphical data.Broader Impact. This research will alter vocal communication interaction in online environments and in physical co-located spaces. The act of archiving in real-time will have new consequences for voice communication and collaboration. The research will provide novel tools for helping users learn the subtleties of vocalization. These tools will be used for learning new languages and for speech therapy. The research extends the vocalization tools to tangible toy objects to encourage vocalization in children who have social impairments. The research will provide an alternate graphical method for archiving large bodies of audio that can be searched at a glance rather than with a search engine. The research results will be disseminated in new courses and in research publications.