Real-time Tongue from Ultrasound

*我们研究的动机是开发一种舌头可视化工具，以帮助需要语言治疗的人、学习外语的人，或者希望“看看他们是如何说话的”的人。舌头是最重要的发音器官，对舌头运动的研究有很大一部分集中在舌头在言语中的作用。听觉训练应用很多，但视觉训练还不成熟。不幸的是，舌头是一个很难看到的器官，使用传统的相机或光学运动跟踪器很难跟踪。这导致了使用核磁共振和超声波机对舌头运动进行可视化。核磁共振图像的质量令人印象深刻，但它的成本高得令人望而却步，而且通常只能在嘈杂和受限的环境中操作。虽然超声图像质量不高，但由于其具有无创测量、设备价格实惠、便携、可实时显示舌头等强大优势，在语音学研究和医学成像领域受到越来越多的欢迎。我们建议使用深度学习和其他最新的机器学习技术来开发一种新的算法，用于从运动舌体的超声成像中高效、稳健和实时地分割舌体。这将导致仅基于超声探头捕获的舌体的单个矢状面角度的3D舌体重建和模拟。我们以前已经从半自动分割的舌部超声图像中执行了静态3D重建，但目前的建议侧重于实时舌部分割。该结果将通过与标准语言说话者的舌头运动进行比较，提供发音错误的视觉反馈。

***The motivation for our research is to develop a tongue visualization tool to help people who require speech therapy, who learn foreign languages, or who otherwise wish to “see how they talk”. The tongue is the most important articulator of speech and a large proportion of the research on tongue movement focuses on its role in speech. Many aural training applications exist but visual training is not mature. Unfortunately, the tongue is an organ difficult to see and hard to track using conventional camera or optical motion trackers. This has led to tongue motion visualization using MRI and ultrasound machines. The quality of MRI images is impressive, but it is prohibitively expensive and generally operates only in a noisy and constrained environment. While ultrasound does not have as high quality images, it is increasingly popular in phonetics research and medical imaging due to strong advantages such as non-invasive measurement, affordable equipment, portability, and real-time display of the tongue. We propose to use deep learning and other recent machine-learning techniques in developing a new algorithm for efficient, robust and real-time tongue segmentation from ultrasound imaging of the tongue in motion. This will lead to 3D tongue reconstruction and simulation, based only on a single sagittal view angle of the tongue captured by a ultrasound probe. We have previously performed static 3D reconstructions from semi-automatically segmented tongue ultrasound images but the present proposal focuses on real-time tongue segmentation. The result will provide visual feedback of mispronunciation through comparison with the tongue motion of a standard language speaker.