Data-based analysis of speech and gesture: the Bielefeld Speech and Gesture Alignment corpus (SaGA) and its applications
Data-based analysis of speech and gesture: the Bielefeld Speech and Gesture Alignment corpus (SaGA) and its applications
复制标题
DOI：
10.1007/s12193-012-0106-8
复制
发表时间：
2012-08
影响因子：
2.9
通讯作者：
Andy Lücking;K. Bergmann;Florian Hahn;S. Kopp;H. Rieser
中科院分区：
文献类型：
--
作者：
Andy Lücking;K. Bergmann;Florian Hahn;S. Kopp;H. Rieser
在面对面的交流中，对话者经常产生由言语和伴随的手势组成的多模态意义包。我们讨论了一个系统的注释语音和手势语料库，包括25个路线和地标描述对话，比勒费尔德语音和手势对齐语料库（佐贺），收集在实验面对面的设置。我们首先描述了语料库的主要和次要数据及其可靠性评估。然后，我们进入一些项目进行使用佐贺展示其广泛的可用性：在经验方面，有工作的手势类型学，个人和上下文参数影响手势生产和手势的功能对话结构。已经建立了扩展基于统一的语法的语音-手势接口。此外，语音-手势对齐的计算模型的开发及其实现构成了我们关注的研究方向。
Communicating face-to-face, interlocutors frequently produce multimodal meaning packages consisting of speech and accompanying gestures. We discuss a systematically annotated speech and gesture corpus consisting of 25 route-and-landmark-description dialogues, the Bielefeld Speech and Gesture Alignment corpus (SaGA), collected in experimental face-to-face settings. We first describe the primary and secondary data of the corpus and its reliability assessment. Then we go into some of the projects carried out using SaGA demonstrating the wide range of its usability: on the empirical side, there is work on gesture typology, individual and contextual parameters influencing gesture production and gestures’ functions for dialogue structure. Speech-gesture interfaces have been established extending unification-based grammars. In addition, the development of a computational model of speech-gesture alignment and its implementation constitutes a research line we focus on.