3D Avatar Approach for Continuous Sign Movement Using Speech/Text
3D Avatar Approach for Continuous Sign Movement Using Speech/Text
复制标题
DOI：
10.3390/app11083439
复制
发表时间：
2021-04-01
影响因子：
2.7
通讯作者：
Kim, Byung-Gyu
中科院分区：
文献类型：
--
作者：
Das Chakladar, Debashis;Kumar, Pradeep;Kim, Byung-Gyu
手语是一种视觉语言，由听力障碍者借助手和手指的动作进行交流。印度手语（Indian Sign Language，ISL）是生活在印度的听障人士的一种发达和标准的交流方式。然而，由于缺乏手语知识，使用口语的其他人在与听力受损者交流时总是面临困难。在这项研究中，我们已经开发了一个基于3D化身的手语学习系统，将输入的语音/文本转换成相应的手语动作。该系统由三个模块组成。最初，输入语音被转换成英语句子。然后，使用自然语言处理（NLP）技术将该英语句子转换为相应的ISL句子。最后，基于ISL语句定义3D化身的运动。翻译模块达到10.50 SER（符号错误率）分数。
Sign language is a visual language for communication used by hearing-impaired people with the help of hand and finger movements. Indian Sign Language (ISL) is a well-developed and standard way of communication for hearing-impaired people living in India. However, other people who use spoken language always face difficulty while communicating with a hearing-impaired person due to lack of sign language knowledge. In this study, we have developed a 3D avatar-based sign language learning system that converts the input speech/text into corresponding sign movements for ISL. The system consists of three modules. Initially, the input speech is converted into an English sentence. Then, that English sentence is converted into the corresponding ISL sentence using the Natural Language Processing (NLP) technique. Finally, the motion of the 3D avatar is defined based on the ISL sentence. The translation module achieves a 10.50 SER (Sign Error Rate) score.