权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Quantification of Gesture Form Analysis (quantitative GFA)Application of GFA to gestures within sign language and development of an interdisciplinary coding system

手势形式分析的量化（定量 GFA）GFA 在手语手势中的应用以及跨学科编码系统的开发

基本信息

批准号：
281272438
负责人：
Dr. Julius Hassemer
金额：
--
依托单位：
Faculdade de Filosofia, Letras e Ciências Humanas
依托单位国家：
德国
项目类别：
Research Fellowships
财政年份：
2015
资助国家：
德国
起止时间：
2014-12-31 至 2016-12-31
项目状态：
已结题

来源：
https://gepris.dfg.de/gepris/projekt/281272438?language=en
关键词：
Quantification Gesture Form Analysis quantitative

项目摘要

Gesture studies, a young but dynamic interdisciplinary area of research of interest to psychology, anthropology, linguistics and informatics, has not yet settled on a common typology of gestures, nor even on the principles on which such a typology might be constructed. It is agreed that co-verbal gestures convey meaning through the physical form and movement of the hand, often by means of iconically representing aspects of objects or imitating practical actions (e.g., Calbris 2003; Cienki 2005; Efron 1941; Kendon 2004; Kita et al. 2007; Mittelberg 2010; Müller 2008; Streeck 2009). But existing categorisations fundamentally disagree on the number of types and on their relations to one another. The number of gesture type categories ranges from 4 (Müller 1998) to 12 (Streeck 2008) to 84 (Sowa 2006), often making cross-studies comparison difficult.The theory of Gesture Form Analysis (GFA), proposes a systematic, form-based categorisation framework, combining insights from the above-mentioned scholars. Basic assumptions of GFA have already been empirically supported in a motion-capture study (Hassemer, dissertation).Still lacking, however, are (1) application of GFA to the gestural elements of sign language and (2) standardisation of a practical coding procedure that allows quantifying gesture data by type across different corpora. These are the two primary foci of this proposal. A third, exploratory goal is to begin testing the viability of GFA as a model for automatic gesture recognition.In support of these goals, we propose to continue the study at the University of São Paulo, with Prof. Leland McCleary, a linguist who is familiar with GFA and who, with his colleague Evani Viotti, has been studying gestural elements of Brazilian Sign Language as well as methods of sign transcription; and whose colleague Felipe Barbosa does research on phonetic variation in sign language, using VICON motion-capture technology. The proposed situation will also permit the initial investigation of GFA applications to machine-learning methods together with a specialist in that area, Sarajane Peres. All of these researchers at the University of São Paulo have agreed to collaborate.Specifically in support of the second goal of implementation of a practical coding method, we bring to the project prior experience of having successfully collaborated (with Han Sloetjes, MPI Nijmegen) on the creation of a gesture annotation plugin for the ELAN transcription software. In support of the second and third goal, we also bring a motion-capture corpus, collected during the dissertation, containing multiple-camera (high-speed) video recordings and three-dimensional motion-capture data on verbal-gestural descriptions that can be used as a show-case corpus for quantitative GFA and for the exploratory machine-learning investigation.

手势研究是一个年轻但充满活力的跨学科研究领域，对心理学、人类学、语言学和信息学都很感兴趣，但它尚未确定一个共同的手势类型学，甚至还没有确定构建这种类型学的原则。人们一致认为，共同语言手势通过手的物理形式和运动来传达意义，通常是通过象征性地表示物体的各个方面或模仿实际动作（例如，Calbris 2003; Cienki 2005; Efron 1941; Kendon 2004; Kita et al. 2007; Mittelberg 2010; m<s:1> ller 2008; Streeck 2009）。但现有的分类在类型的数量和它们之间的关系上根本不一致。手势类型类别的数量从4个（m<s:1> ller 1998）到12个（Streeck 2008）到84个（Sowa 2006）不等，这往往使交叉研究比较变得困难。手势形式分析理论（GFA）结合上述学者的见解，提出了一个系统的、基于形式的分类框架。GFA的基本假设已经在一项动作捕捉研究中得到了实证支持（haassemer，论文）。然而，仍然缺乏的是(1)将GFA应用于手语的手势元素和(2)标准化的实用编码程序，该程序允许在不同的语料库中按类型量化手势数据。这是本提案的两个主要焦点。第三个探索性目标是开始测试GFA作为自动手势识别模型的可行性。为了支持这些目标，我们建议在<s:1>圣保罗大学继续研究，Leland McCleary教授是一位熟悉GFA的语言学家，他和他的同事Evani Viotti一直在研究巴西手语的手势元素以及符号转录方法；他的同事费利佩·巴博萨（Felipe Barbosa）使用VICON动作捕捉技术研究手语的语音变化。拟议的情况还将允许与该领域的专家Sarajane Peres一起对GFA在机器学习方法中的应用进行初步调查。<s:1>圣保罗大学的所有这些研究人员都同意合作。特别是为了支持实现实用编码方法的第二个目标，我们为项目带来了之前成功合作的经验（与Han Sloetjes， MPI Nijmegen），为ELAN转录软件创建手势注释插件。为了支持第二个和第三个目标，我们还带来了一个在论文中收集的动作捕捉语料库，其中包含多摄像机（高速）视频记录和关于语言手势描述的三维动作捕捉数据，可以用作定量GFA和探索性机器学习调查的展示语料库。