ExTOL: End to End Translation of British Sign Language

ExTOL：英国手语的端到端翻译

• 批准号: EP/R03298X/1
• 负责人: Richard Bowden
• 金额: $ 123.84万
• 依托单位: University of Surrey
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2018
• 资助国家: 英国
• 起止时间: 2018 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FR03298X%2F1
• 关键词: ExTOL; End; Translation; British; Sign

British Sign Language (BSL) is the natural language of the British Deaf community and is as rich and expressive as any spoken language. However, BSL is not just English words converted into hand motions. It is a language in its own right, with its own grammar, very different from English. Also BSL uses different elements of the body simultaneously. Not just the movement and shape of the hands but the body, face, mouth and the space around the signer are all used to convey meaning. Linguistic study of sign languages is quite new compared to spoken languages, having begun only in the 1960s. Linguists are very interested in sign languages because of what they can reveal about the possibilities of human language that don't rely at all on sound. One of the problems is that studying sign languages involves analysing video footage - and because sign languages lack any standard writing or transcription system, this is extremely labour-intensive. This project will develop computer vision tools to assist with video analysis. This will in turn help linguists increase their knowledge of the language with a long term ambition of creating the world's first machine readable dataset of a sign language, a goal that was achieved for large amounts of text of spoken language in the 1970s. The ultimate goal of this project is to take the annotated data and understanding from linguistic study and to use this to build a system that is capable of watching a human signing and turning this into written English. This will be a world first and an important landmark for deaf-hearing communication. To achieve this the computer must be able to recognise not only hand motion and shape but the facial expression and body posture of the signer. It must also understanding how these aspects are put together into phrases and how these can be translated into written/spoken language.Although there have been some recent advances in sign language recognition via data gloves and motion capture systems like Kinect, part of the problem is that most computer scientists in this research area do not have the required in-depth knowledge of sign language. This project is therefore a strategic collaboration between leading experts in British Sign Language linguistics and software engineers who specialise in computer vision and machine learning, with the aim of building the world's first British Sign Language to English Translation system and the first practically functional machine translation system for any sign language.

英国手语 (BSL) 是英国聋人社区的自然语言，与任何口语一样丰富且富有表现力。然而，BSL 不仅仅是将英语单词转换成手势。它本身就是一种语言，有自己的语法，与英语非常不同。 BSL 同时使用身体的不同元素。不仅是手的动作和形状，还有身体、脸部、嘴巴以及签名者周围的空间都用来传达含义。与口语相比，手语的语言学研究相当新，直到 20 世纪 60 年代才开始。语言学家对手语非常感兴趣，因为它们可以揭示完全不依赖声音的人类语言的可能性。问题之一是研究手语需要分析视频片段，而且由于手语缺乏任何标准的书写或转录系统，因此这是极其耗费人力的。该项目将开发计算机视觉工具来协助视频分析。这反过来将帮助语言学家增加他们对语言的了解，长期目标是创建世界上第一个机器可读的手语数据集，这一目标在 20 世纪 70 年代针对大量口语文本实现了。该项目的最终目标是从语言研究中获取带注释的数据和理解，并使用它来构建一个能够观看人类手势并将其转化为书面英语的系统。这将是世界首创，也是聋人交流的重要里程碑。为了实现这一目标，计算机不仅必须能够识别手部动作和形状，还必须能够识别签名者的面部表情和身体姿势。它还必须了解如何将这些方面组合成短语以及如何将它们翻译成书面/口语。尽管最近通过数据手套和 Kinect 等动作捕捉系统在手语识别方面取得了一些进展，但部分问题是该研究领域的大多数计算机科学家不具备所需的手语深入知识。因此，该项目是英国手语语言学领域的顶尖专家与计算机视觉和机器学习领域的软件工程师之间的战略合作，旨在构建世界上第一个英国手语到英语的翻译系统以及第一个适用于任何手语的实用机器翻译系统。

项目成果

Content4All Open Research Sign Language Translation Datasets

• DOI: 10.1109/fg52635.2021.9667087
• 发表时间: 2021-05
• 期刊: 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021)
• 作者: N. C. Camgoz;Ben Saunders;Guillaume Rochette;Marco Giovanelli;Giacomo Inches;Robin Nachtrab-Ribback;R. Bowden

BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues

• DOI: 10.1007/978-3-030-58621-8_3
• 发表时间: 2020-07
• 期刊: ArXiv
• 作者: Samuel Albanie;Gül Varol;Liliane Momeni;Triantafyllos Afouras;Joon Son Chung;Neil Fox;Andrew Zisserman

SeeHear: Signer Diarisation and a New Dataset

• DOI: 10.1109/icassp39728.2021.9414856
• 发表时间: 2021-06
• 期刊: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• 作者: Samuel Albanie;Gül Varol;Liliane Momeni;Triantafyllos Afouras;Andrew Brown;Chuhan Zhang;Ernesto Coto;Necati Cihan Camgöz;Ben Saunders;Abhishek Dutta;Neil Fox;R. Bowden;B. Woll;Andrew Zisserman

Computer Vision - ECCV 2020 Workshops - Glasgow, UK, August 23-28, 2020, Proceedings, Part II

计算机视觉 - ECCV 2020 研讨会 - 英国格拉斯哥，2020 年 8 月 23-28 日，会议记录，第二部分

• DOI: 10.1007/978-3-030-66096-3_13
• 发表时间: 2020
• 作者: Camgöz N

SLRTP 2020: The Sign Language Recognition, Translation & Production Workshop

SLRTP 2020：手语识别、翻译

• 发表时间: 2020
• 作者: Camgoz, NC