A Study Examining a Real-Time Sign Language-to-Text Interpretation System Using Crowdsourcing
A Study Examining a Real-Time Sign Language-to-Text Interpretation System Using Crowdsourcing
复制标题
使用众包检查实时手语到文本翻译系统的研究
DOI：
10.1007/978-3-030-58805-2_22
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Minagawa Hiroki
中科院分区：
文献类型：
--
作者：
Tanaka Kohei;Wakatsuki Daisuke;Minagawa Hiroki
当前的研究探讨了如何使用众包将手语转换为文本。一般来说，在日本，手语翻译员会读出说话者的手语并发出声音，字幕打字员会根据发声生成字幕。然而，这种方法使劳动力成本增加了一倍，并且延迟了字幕的提供。因此，我们开发了一种通过众包将手语翻译为字幕文本的系统，由非专家进行翻译。虽然许多可以阅读手语的聋人/听力障碍者 (DHH) 适合这项任务，但并非所有人都具备足够的打字技能。为了解决这个问题，我们的系统将实时手语视频分成较短的片段，并将其分发给工作人员。工作人员将片段解释并输入文本后，系统通过整合这些文本生成字幕。此外，我们还提供了一个用于播放速度控制和一秒倒带的用户界面，以提高任务完成的便捷性。我们的系统可以建立一个环境，不仅可以将手语翻译为字幕文本，还可以为 DHH 个人提供帮助那些无法阅读手语的人的机会。我们使用我们的原型系统进行了手语到文本翻译的测试。对于 9 秒的分段，工作人员完成任务的平均时间为 26 秒。文本缺失和片段之间冲突的合计率为 66%。对问卷调查结果的分析发现，分配较少任务的员工认为这些任务更令人愉快。
The current study examined how to use crowdsourcing to convert sign language-to-text. Generally in Japan, a sign language interpreter reads and vocalizes the sign language of the speaker, and caption typists generate captions from the vocalization. However, this method doubles labor costs and delays caption provision. Therefore, we developed a system that interprets sign language-to-caption text via crowdsourcing, with non-experts performing interpretations. While many individuals classified as deaf/hard-of-hearing (DHH) who can read sign language are suitable for this task, not all of them possess adequate typing skills. To address this, our system divides live sign language video into shorter segments, distributing them to workers. After the worker interprets and types the segments to text, the system generates captions through integration of these texts. Furthermore, we provide a user interface for playback speed control and one second rewinding in order to improve the ease with which tasks are completed. Our system can establish an environment that not only allows the interpretation of sign language-to-caption text, but also provides an opportunity for DHH individuals to assist those that are unable read sign language. We conducted a test using our prototype system for sign language-to-text interpretation. The mean time it took a worker to finish a task was 26 s for a 9 s segment. The combined total rate of missing text and collision between segments was 66%. Analysis of questionnaire responses found that workers assigned fewer tasks considered the tasks more enjoyable.