ExTOL: End to End Translation of British Sign Language

英国手语(BSL)是英国聋人社区的自然语言，与任何口语一样丰富和富有表现力。然而，BSL不仅仅是转换成手势的英语单词。英语本身就是一门语言，有自己的语法，与英语截然不同。此外，BSL同时使用身体的不同元素。不仅是手的动作和形状，还有身体、面部、嘴巴和签名者周围的空间都用来传达意思。与口语相比，手语的语言学研究是相当新的，直到20世纪60年代才开始。语言学家对手语非常感兴趣，因为它们可以揭示出人类语言完全不依赖于声音的可能性。其中一个问题是，学习手语涉及到分析视频片段--由于手语缺乏任何标准的书写或转录系统，这是极其劳动密集型的。该项目将开发计算机视觉工具，以协助视频分析。这反过来将帮助语言学家增加他们对手语的知识，并雄心勃勃地创建世界上第一个机器可读的手语数据集，这一目标在20世纪70年代实现了对大量口语文本的研究。这个项目的最终目标是从语言学研究中获得带注释的数据和理解，并利用这些数据建立一个能够观看人类手势并将其转化为书面英语的系统。这将是世界上第一个，也是聋人听力交流的一个重要里程碑。要做到这一点，计算机不仅必须能够识别手部动作和形状，还必须能够识别签名者的面部表情和身体姿势。它还必须了解这些方面是如何组合成短语，以及如何将这些短语翻译成书面/口头语言。尽管最近通过数据手套和Kinect等动作捕捉系统在手语识别方面取得了一些进展，但部分问题是，这一研究领域的大多数计算机科学家没有必要的手语深入知识。因此，该项目是英国手语语言学的顶尖专家和专门从事计算机视觉和机器学习的软件工程师之间的战略合作，旨在建立世界上第一个英国手语到英语的翻译系统，以及第一个实用的任何手语的机器翻译系统。

British Sign Language (BSL) is the natural language of the British Deaf community and is as rich and expressive as any spoken language. However, BSL is not just English words converted into hand motions. It is a language in its own right, with its own grammar, very different from English. Also BSL uses different elements of the body simultaneously. Not just the movement and shape of the hands but the body, face, mouth and the space around the signer are all used to convey meaning. Linguistic study of sign languages is quite new compared to spoken languages, having begun only in the 1960s. Linguists are very interested in sign languages because of what they can reveal about the possibilities of human language that don't rely at all on sound. One of the problems is that studying sign languages involves analysing video footage - and because sign languages lack any standard writing or transcription system, this is extremely labour-intensive. This project will develop computer vision tools to assist with video analysis. This will in turn help linguists increase their knowledge of the language with a long term ambition of creating the world's first machine readable dataset of a sign language, a goal that was achieved for large amounts of text of spoken language in the 1970s. The ultimate goal of this project is to take the annotated data and understanding from linguistic study and to use this to build a system that is capable of watching a human signing and turning this into written English. This will be a world first and an important landmark for deaf-hearing communication. To achieve this the computer must be able to recognise not only hand motion and shape but the facial expression and body posture of the signer. It must also understanding how these aspects are put together into phrases and how these can be translated into written/spoken language.Although there have been some recent advances in sign language recognition via data gloves and motion capture systems like Kinect, part of the problem is that most computer scientists in this research area do not have the required in-depth knowledge of sign language. This project is therefore a strategic collaboration between leading experts in British Sign Language linguistics and software engineers who specialise in computer vision and machine learning, with the aim of building the world's first British Sign Language to English Translation system and the first practically functional machine translation system for any sign language.