Video accessibility enhancement for hearing-impaired users
Video accessibility enhancement for hearing-impaired users
复制标题
DOI：
10.1145/2037676.2037681
复制
发表时间：
2011-10
期刊：
影响因子：
--
通讯作者：
Richang Hong;Meng Wang;Xiao-Tong Yuan;Mengdi Xu;Jianguo Jiang;Shuicheng Yan;Tat-Seng Chua
中科院分区：
文献类型：
--
作者：
Richang Hong;Meng Wang;Xiao-Tong Yuan;Mengdi Xu;Jianguo Jiang;Shuicheng Yan;Tat-Seng Chua
全球有超过6600万人患有听力障碍，由于音频信息的丢失，这种残疾给他们理解视频内容带来了困难。如果有脚本，字幕技术可以在视频播放过程中同步显示脚本，在一定程度上帮助他们。然而，我们发现，现有的字幕技术是远远不能令人满意的，在协助听障观众欣赏视频。在这篇文章中，我们介绍了一个计划，以提高视频无障碍使用动态字幕的方法，它探讨了丰富的技术，包括人脸检测和识别，视觉显著性分析，文本语音对齐，等不同于现有的方法，被归类为静态字幕，动态字幕把脚本在合适的位置，以帮助听障观众更好地认识说话的字符。此外，它逐步突出的脚本，通过对齐他们与语音信号，并说明了音量的变化。通过这种方式，特殊观众可以更好地跟踪脚本，并感知音量变化所传达的情绪。我们在20个视频片段上实现了该技术，并与60名真实的听障用户进行了深入的研究。实验结果证明了视频无障碍增强方案的有效性和实用性。
There are more than 66 million people suffering from hearing impairment and this disability brings them difficulty in video content understanding due to the loss of audio information. If the scripts are available, captioning technology can help them in a certain degree by synchronously illustrating the scripts during the playing of videos. However, we show that the existing captioning techniques are far from satisfactory in assisting the hearing-impaired audience to enjoy videos. In this article, we introduce a scheme to enhance video accessibility using a Dynamic Captioning approach, which explores a rich set of technologies including face detection and recognition, visual saliency analysis, text-speech alignment, etc. Different from the existing methods that are categorized as static captioning, dynamic captioning puts scripts at suitable positions to help the hearing-impaired audience better recognize the speaking characters. In addition, it progressively highlights the scripts word-by-word via aligning them with the speech signal and illustrates the variation of voice volume. In this way, the special audience can better track the scripts and perceive the moods that are conveyed by the variation of volume. We implemented the technology on 20 video clips and conducted an in-depth study with 60 real hearing-impaired users. The results demonstrated the effectiveness and usefulness of the video accessibility enhancement scheme.