Towards Fully Automated Motion Capture of Signs - Development and Evaluation of a Key Word Signing Avatar
Towards Fully Automated Motion Capture of Signs - Development and Evaluation of a Key Word Signing Avatar
复制标题
DOI：
10.1145/2764918
复制
发表时间：
2015-07-01
影响因子：
2.4
通讯作者：
Beskow, Jonas
中科院分区：
文献类型：
--
作者：
Alexanderson, Simon;Beskow, Jonas
标志的动作捕捉为多模式数据收集领域带来了独特的挑战。视觉信息的密集包装需要捕获数据的高保真度和高带宽。尽管基于标记的光学运动捕捉提供了许多理想的功能，例如高精度、全局拟合以及同时记录身体和面部的能力，但它并没有广泛用于记录手指运动，特别是对于手势等铰接和句法运动。相反，大多数签名化身项目都使用昂贵的仪器手套，这需要很长的校准程序。在本文中，我们评估了使用大量低成本相机对瑞典手语中的孤立标志进行光学运动捕捉所获得的数据质量。我们还提出了一种新颖的双传感器方法，将数据与低成本的五传感器仪表手套相结合，提供一种低手动后处理的记录方法。最后，我们评估收集的数据和双传感器方法，并将其转移到高度风格化的化身上。该虚拟形象的应用是一个基于游戏的环境，用于训练关键词手语（KWS）作为增强和替代沟通（AAC），专门针对有沟通障碍的儿童。
Motion capture of signs provides unique challenges in the field of multimodal data collection. The dense packaging of visual information requires high fidelity and high bandwidth of the captured data. Even though marker-based optical motion capture provides many desirable features such as high accuracy, global fitting, and the ability to record body and face simultaneously, it is not widely used to record finger motion, especially not for articulated and syntactic motion such as signs. Instead, most signing avatar projects use costly instrumented gloves, which require long calibration procedures. In this article, we evaluate the data quality obtained from optical motion capture of isolated signs from Swedish sign language with a large number of low-cost cameras. We also present a novel dual-sensor approach to combine the data with low-cost, five-sensor instrumented gloves to provide a recording method with low manual postprocessing. Finally, we evaluate the collected data and the dual-sensor approach as transferred to a highly stylized avatar. The application of the avatar is a game-based environment for training Key Word Signing (KWS) as augmented and alternative communication (AAC), intended for children with communication disabilities.