Deep CNN-Based Recognition of JSL Finger Spelling
Deep CNN-Based Recognition of JSL Finger Spelling
复制标题
DOI：
10.1007/978-3-030-29859-3_51
复制
发表时间：
2019-09
期刊：
影响因子：
--
通讯作者：
Nam Tu Nguen;Shinji Sako;B. Kwolek
中科院分区：
文献类型：
--
作者：
Nam Tu Nguen;Shinji Sako;B. Kwolek
在本文中，我们提出了一个在 RGB 图像上识别日语手语静态手指拼写的框架。手指拼写符号由一个基于 ResNet 的卷积神经网络和两个 ResNet 四元数卷积神经网络组成的集成体进行识别。 3D 关节手模型已用于生成合成手指拼写并扩展由真实手势组成的数据集。为 41 个手势中的每一个手势准备了 12 种不同的手势实现。通过起始姿势和结束姿势之间的插值，为每个实现渲染了十张图像。实验结果表明，由于足够多的训练数据，单个 RGB 相机的图像可以获得很高的识别率。 ResNet四元数卷积神经网络取得的结果优于ResNet CNN取得的结果。最好的识别结果是由集成实现的。 JSL-rend 数据集可供下载。
In this paper, we present a framework for recognition of static finger spelling in Japanese Sign Language on RGB images. The finger spelled signs were recognized by an ensemble consisting of a ResNet-based convolutional neural network and two ResNet quaternion convolutional neural networks. A 3D articulated hand model has been used to generate synthetic finger spellings and to extend a dataset consisting of real hand gestures. Twelve different gesture realizations were prepared for each of 41 signs. Ten images have been rendered for each realization through interpolations between the starting and end poses. Experimental results demonstrate that owing to sufficient amount of training data a high recognition rate can be attained on images from a single RGB camera. Results achieved by the ResNet quaternion convolutional neural network are better than results obtained by the ResNet CNN. The best recognition results were achieved by the ensemble. The JSL-rend dataset is available for download.