WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language
WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language
复制标题
WLASL-LEX：用于识别美国手语语音属性的数据集
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
A. Cangelosi
中科院分区：
文献类型：
--
作者：
Federico Tavella;Viktor Schlegel;M. Romeo;Aphrodite Galata;A. Cangelosi
手语处理（SLP）涉及手语的自动化处理，手语是聋人和听力受损者的主要交流方式。SLP具有许多不同的任务，从符号识别到翻译和手语的产生，但迄今为止一直被NLP社区所忽视。我们利用现有的资源，构建了一个大规模的美国手语符号数据集，注释了六个不同的语音属性。然后，我们进行了广泛的实证研究，以调查数据驱动的端到端和基于特征的方法是否可以优化，以自动识别这些属性。我们发现，尽管这项任务存在固有的挑战，但基于图的神经网络在从原始视频中提取的骨架特征上运行，能够在不同程度上成功完成任务。最重要的是，我们表明，这种性能甚至涉及在训练过程中未观察到的迹象。
Signed Language Processing (SLP) concerns the automated processing of signed languages, the main means of communication of Deaf and hearing impaired individuals. SLP features many different tasks, ranging from sign recognition to translation and production of signed speech, but has been overlooked by the NLP community thus far.In this paper, we bring to attention the task of modelling the phonology of sign languages. We leverage existing resources to construct a large-scale dataset of American Sign Language signs annotated with six different phonological properties. We then conduct an extensive empirical study to investigate whether data-driven end-to-end and feature-based approaches can be optimised to automatically recognise these properties. We find that, despite the inherent challenges of the task, graph-based neural networks that operate over skeleton features extracted from raw videos are able to succeed at the task to a varying degree. Most importantly, we show that this performance pertains even on signs unobserved during training.