LILiR2 - Language Independent Lip Reading

众所周知，人类能够读唇语，而且确实会读唇语，但对于有效读唇语到底需要哪些视觉信息，尤其是在非实验室环境中，我们所知甚少。这个项目将收集唇读数据，并用它来建立自动唇读系统：将唇读视频转换为文本的机器。为了有效，这些系统必须准确地跟踪头部的各种姿势；提取描述嘴唇的数字或特征，然后学习哪些特征对应于哪些文本。为了解决这个问题，我们需要使用从语音中收集的信息。因此，本项目还将研究如何利用已知的大量音频语音信息来识别视觉语音。该项目是东安格利亚大学（University of East Anglia）之间的合作，该大学此前开发了最先进的语音阅读系统；萨里大学制造了准确可靠的面部和嘴唇追踪器，内政部科学部门希望调查这种方法在打击犯罪方面的可行性。

It is known that humans can, and do, lip-read but not much is known about exactly what visual information is needed for effective lip-reading, particularly in non-laboratory environments. This project will collect data for lip-reading and use it to build automatic lip-reading systems: machines that convert videos of lip-motions into text. To be effective such systems must accurately track the head over a variety of poses; extract numbers, or features, that describe the lips and then learn what features correspond to what text. To tackle the problem we will need to use information collected from audio speech. So this project will also investigate how to use the extensive information known about audio speech to recognise visual speech.The project is a collaboration between the University of East Anglia who have previously developed state-of-the-art speech reading systems; the University of Surrey who built accurate and reliable face and lip-trackers and the Home Office Scientific Branch who wish to investigate the feasibility of this approach for crime fighting.