Real-time Tongue from Ultrasound

点击翻译按钮获取中文摘要

***The motivation for our research is to develop a tongue visualization tool to help people who require speech therapy, who learn foreign languages, or who otherwise wish to “see how they talk”. The tongue is the most important articulator of speech and a large proportion of the research on tongue movement focuses on its role in speech. Many aural training applications exist but visual training is not mature. Unfortunately, the tongue is an organ difficult to see and hard to track using conventional camera or optical motion trackers. This has led to tongue motion visualization using MRI and ultrasound machines. The quality of MRI images is impressive, but it is prohibitively expensive and generally operates only in a noisy and constrained environment. While ultrasound does not have as high quality images, it is increasingly popular in phonetics research and medical imaging due to strong advantages such as non-invasive measurement, affordable equipment, portability, and real-time display of the tongue. We propose to use deep learning and other recent machine-learning techniques in developing a new algorithm for efficient, robust and real-time tongue segmentation from ultrasound imaging of the tongue in motion. This will lead to 3D tongue reconstruction and simulation, based only on a single sagittal view angle of the tongue captured by a ultrasound probe. We have previously performed static 3D reconstructions from semi-automatically segmented tongue ultrasound images but the present proposal focuses on real-time tongue segmentation. The result will provide visual feedback of mispronunciation through comparison with the tongue motion of a standard language speaker.