Visual Solutions for Automated Translation Between Spoken and Signed Languages

口语和手语之间自动翻译的可视化解决方案。我们建议建立一个稳健的视觉语音识别系统，分析口语图像，并实现至少具有人类专家识别率的话语识别。这个视觉语音识别系统随后将与我们现有的手势识别系统集成，以提高性能，就像人类将视觉和音频数据结合起来进行语言理解一样。其结果将是一个在实际应用领域提供英语和澳大利亚手语Auslan之间的翻译的系统。值得注意的是，我们的工作将提供对将语言和手势联系起来的神经活动的认知模型的见解。

Visual Solutions for Automated Translation Between Spoken and Signed Languages. We propose to build a robust visual speech recognition system that analyzes images of spoken language and achieves a recognition of the utterances with at least human expert recognition rates. This visual speech recognition system will then be integrated with our existing gesture recognition system to improve performance, just as humans combine visual and audio data for language understanding. The result will be a system providing translation between English and the Australian sign language Auslan in a practical application domain. Significantly, our work will provide insights into the cognitive models of neural activity linking language and gesture.