Large-scale Learning of Sign Language by Watching TV (Using Co-occurrences)
Large-scale Learning of Sign Language by Watching TV (Using Co-occurrences)
复制标题
DOI：
10.5244/c.27.20
复制
发表时间：
2013
期刊：
影响因子：
--
通讯作者：
Tomas Pfister;James Charles;Andrew Zisserman
中科院分区：
文献类型：
--
作者：
Tomas Pfister;James Charles;Andrew Zisserman
这项工作的目标是从手语翻译的电视广播中自动学习大量的符号。我们实现这一点，利用监管信息中的字幕的广播。然而，该信息既弱又有噪声，并且当试图识别符号的时间窗口时，这导致具有挑战性的对应问题。我们做出了以下贡献：（i）我们表明，有些反直觉，嘴型是高度信息隔离的话，在一种语言的聋人，和他们的共同出现与签署可以用来显着减少对应的搜索空间;以及（ii）我们开发了一种使用有效判别搜索的多实例学习方法，其确定具有高查全率和精确率的符号的候选列表。我们演示的方法从BBC电视广播的视频，并实现更高的准确率和召回比以前的方法，尽管使用更简单的功能。
The goal of this work is to automatically learn a large number of signs from sign language-interpreted TV broadcasts. We achieve this by exploiting supervisory information available in the subtitles of the broadcasts. However, this information is both weak and noisy and this leads to a challenging correspondence problem when trying to identify the temporal window of the sign. We make the following contributions: (i) we show that, somewhat counter-intuitively, mouth patterns are highly informative for isolating words in a language for the Deaf, and their co-occurrence with signing can be used to significantly reduce the correspondence search space; and (ii) we develop a multiple instance learning method using an efficient discriminative search, which determines a candidate list for the sign with both high recall and precision. We demonstrate the method on videos from BBC TV broadcasts, and achieve higher accuracy and recall than previous methods, despite using much simpler features.