3D Hands, Face and Body Extraction for Sign Language Recognition
3D Hands, Face and Body Extraction for Sign Language Recognition
复制标题
用于手语识别的 3D 手部、面部和身体提取
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
P. Maragos
中科院分区：
文献类型：
--
作者：
Agelos Kratimenos;G. Pavlakos;P. Maragos
。对于手语识别(SLR)问题，大部分信息包括三个主要渠道：手势、面部表情和身体姿势。虽然许多最先进的作品已经成功地独立深入阐述了这些功能，但就我们所知，还没有作品充分结合这三个信息渠道，特别是在3D中，来科学地识别手语。ffi在这项工作中，我们使用了SMPL-X，这是一种当代的参数模型，可以从一幅图像中联合提取3D身体形状、面部和手的信息。我们将这种整体3D重建用于单反，证明了它比从原始RGB图像或2D骨骼识别更高的准确性。同时，我们论证了综合来自三个渠道的信息的重要性，以达到最佳的识别效果。
. For the problem of Sign Language Recognition (SLR), the majority of the information is included in three main channels; hand gestures, facial expression and body pose. While many state-of-the-art works have managed to deeply elaborate on these features independently, to the best of our knowledge, no work has adequately combined all these three information channels, particularly in 3D, to efficiently recognize Sign Language. In this work, we employ SMPL-X, a contemporary parametric model that enables joint extraction of 3D body shape, face and hands information from a single image. We use this holistic 3D reconstruction for SLR, demonstrating that it leads to higher accuracy than recognition from raw RGB images, or 2D skeletons. Simultaneously, we demonstrate the importance of combining the information from all three channels, to achieve the best recognition results.