Quantification of Gesture Form Analysis (quantitative GFA)Application of GFA to gestures within sign language and development of an interdisciplinary coding system

手势研究是一个年轻但充满活力的跨学科研究领域，涉及心理学、人类学、语言学和信息学，尚未确定一个共同的手势类型学，甚至还没有确定构建这种类型学的原则。人们一致认为，共同语言手势通过手的物理形式和运动来传达意义，通常是通过形象地表示物体的各个方面或模仿实际动作（例如，Calbris 2003; Cienki 2005; Efron 1941; Kendon 2004; Kita et al. 2007;米特尔贝格2010; Müller 2008; Streeck 2009）。但是，现有的分类在类型的数量和它们之间的关系上根本不一致。手势类型分类的数量从4个（Müller 1998）到12个（Streeck 2008）再到84个（Sowa 2006）不等，这使得交叉研究比较困难。手势形式分析（GFA）理论结合了上述学者的见解，提出了一个系统的、基于形式的分类框架。GFA的基本假设已经在动作捕捉研究中得到了实证支持（Hasemer，论文）。然而，仍然缺乏的是（1）将GFA应用于手语的手势元素和（2）实际编码过程的标准化，允许跨不同语料库按类型量化手势数据。这是本提案的两个主要重点。第三，探索性的目标是开始测试GFA作为自动手势识别模型的可行性，为了支持这些目标，我们建议在圣保罗大学继续这项研究，与Leland McCleary教授一起，他是一位熟悉GFA的语言学家，他和他的同事Evani Viotti一直在研究巴西手语的手势元素以及手语转录的方法;他的同事Felipe巴博萨用VICON动作捕捉技术研究手语中的语音变化。拟议的情况还将允许与该领域的专家Sarajane Peres一起对GFA应用于机器学习方法进行初步调查。圣保罗大学的所有研究人员都同意合作。特别是为了支持实现实用编码方法的第二个目标，我们将之前成功合作（与MPI Nijmegen的Han Sloetjes）为ELAN转录软件创建手势注释插件的经验带入项目。 为了支持第二个和第三个目标，我们还带来了一个运动捕捉语料库，在论文中收集，包含多个摄像头（高速）的视频记录和三维运动捕捉数据的语言，手势的描述，可以用作展示案例语料库的定量GFA和探索性的机器学习调查。

Gesture studies, a young but dynamic interdisciplinary area of research of interest to psychology, anthropology, linguistics and informatics, has not yet settled on a common typology of gestures, nor even on the principles on which such a typology might be constructed. It is agreed that co-verbal gestures convey meaning through the physical form and movement of the hand, often by means of iconically representing aspects of objects or imitating practical actions (e.g., Calbris 2003; Cienki 2005; Efron 1941; Kendon 2004; Kita et al. 2007; Mittelberg 2010; Müller 2008; Streeck 2009). But existing categorisations fundamentally disagree on the number of types and on their relations to one another. The number of gesture type categories ranges from 4 (Müller 1998) to 12 (Streeck 2008) to 84 (Sowa 2006), often making cross-studies comparison difficult.The theory of Gesture Form Analysis (GFA), proposes a systematic, form-based categorisation framework, combining insights from the above-mentioned scholars. Basic assumptions of GFA have already been empirically supported in a motion-capture study (Hassemer, dissertation).Still lacking, however, are (1) application of GFA to the gestural elements of sign language and (2) standardisation of a practical coding procedure that allows quantifying gesture data by type across different corpora. These are the two primary foci of this proposal. A third, exploratory goal is to begin testing the viability of GFA as a model for automatic gesture recognition.In support of these goals, we propose to continue the study at the University of São Paulo, with Prof. Leland McCleary, a linguist who is familiar with GFA and who, with his colleague Evani Viotti, has been studying gestural elements of Brazilian Sign Language as well as methods of sign transcription; and whose colleague Felipe Barbosa does research on phonetic variation in sign language, using VICON motion-capture technology. The proposed situation will also permit the initial investigation of GFA applications to machine-learning methods together with a specialist in that area, Sarajane Peres. All of these researchers at the University of São Paulo have agreed to collaborate.Specifically in support of the second goal of implementation of a practical coding method, we bring to the project prior experience of having successfully collaborated (with Han Sloetjes, MPI Nijmegen) on the creation of a gesture annotation plugin for the ELAN transcription software. In support of the second and third goal, we also bring a motion-capture corpus, collected during the dissertation, containing multiple-camera (high-speed) video recordings and three-dimensional motion-capture data on verbal-gestural descriptions that can be used as a show-case corpus for quantitative GFA and for the exploratory machine-learning investigation.