Ultrax: Real-time tongue tracking for speech therapy using ultrasound

Ultrax：使用超声波进行言语治疗的实时舌头追踪

• 批准号: EP/I027696/1
• 负责人: Steve Renals
• 金额: $ 74.69万
• 依托单位: University of Edinburgh
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2011
• 资助国家: 英国
• 起止时间: 2011 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FI027696%2F1
• 关键词: Ultrax; Real; time; tongue; tracking

Speech Sound Disorders (SSDs) are the most common communication impairment in childhood, affecting 6.5% of all UK children, or 2 children in every classroom. SSDs make it difficult for people to communicate with peers and integrate with society, yet the efficacy of interventions for most types of SSDs is weak. Speech, Language and Communication Disorders (SLCD) are a key UK government priority at present, with 2011 designated the national year of speech, language and communication . A recent government report (Bercow, 2008) highlighted the need for a programme of research to enhance the evidence base for children and young people with SLCD. Our programme of research aims to fulfill this need by developing technology which will aid the assessment, diagnosis and treatment of SSDs. Currently in Speech and Language Therapy, technological support is sparse. Technologies that do exist have been expensive to run or complicated to operate and hence not adopted in clinical practice. This project will develop technology (Ultrax) to turn ultrasound into a tongue imaging device specifically designed to provide real-time visual feedback of tongue movements. Most interventions for SSDs rely heavily on auditory skills; clients must listen to their own productions and modify them. However, with Ultrax people with SSDs will actually be able to see the movements of their own tongues and use this information to modify their speech. It is already possible to capture tongue movements by placing a standard medical ultrasound probe under the chin. Ultrasound has the potential to provide powerful information about atypical speech and to enable speakers to modify their own incorrect articulations. However, the image is grainy, information (especially about the tongue tip) is often lost and the image is difficult to interpret. We will improve this image by exploiting prior knowledge about the range of possible tongue shapes and movements in order to provide valuable constraints in tracking tongue contours in sequences of ultrasound images. We will apply a tongue model to this problem, making use of explicit sequence-based optimization for dynamic tracking and smoothing through time. We will use this technology to enhance the ultrasound images, transforming them into a dynamic, real-time 2D video of the tongue's movements which we hypothesize will be A) more easily understood by children B) extend the range of visible tongue shapes from only vowels and /r/ to include /t/,/k/,/ch/ and other consonants which are often targets for therapy Ultrax will be used to provide bio-feedback therapy for people with SSDs and to provide a means for objectively assessing progress by comparing tongue shapes before and after therapy. We will collect a large database of ultrasound and MRI images of tongue movements from 12 adults (ultrasound and MRI) and 90 primary school children (ultrasound) on which to base the model of tongue contours and to test its performance. At the same time, we will split the 90 children into 3 groups and record each group's response to one of 3 types of ultrasound display: 1. Raw, unenhanced, ultrasound 2. Unenhanced ultrasound with added anatomical context (e.g. the position of the teeth and roof of the mouth) 3. Fully enhanced ultrasound, developed in this project. The ability of the children to imitate tongue shapes and movements will be evaluated to determine whether they find the enhanced images easier to interpret than unenhanced images, leading to an improved ability use ultrasound for bio-visual feedback. We will trial ultrasound therapy with 9 children with SSDs (3 children for each type of display) enabling us to evaluate practical issues arising during therapy and pave the way for a future clinical trial. At the conclusion of our research project we will have developed the basis for a new visual-feedback tool (Ultrax) for Speech and Language Therapists to use in the diagnosis and treatment of SSDs.

语音障碍（ssd）是儿童时期最常见的沟通障碍，影响了6.5%的英国儿童，即每个教室有2个孩子。ssd使患者难以与同伴沟通和融入社会，但对大多数类型ssd的干预效果较弱。言语、语言和交流障碍（SLCD）是英国政府目前的一个关键优先事项，2011年被指定为全国言语、语言和交流年。最近的一份政府报告（Bercow, 2008）强调需要一个研究项目来加强患有SLCD的儿童和青少年的证据基础。我们的研究计划旨在通过开发有助于ssd评估、诊断和治疗的技术来满足这一需求。目前在言语和语言治疗方面，技术支持很少。现有的技术运行成本昂贵或操作复杂，因此没有在临床实践中采用。该项目将开发技术（Ultrax），将超声波转化为舌头成像设备，专门设计用于提供舌头运动的实时视觉反馈。大多数针对固态硬盘的干预措施严重依赖听觉技能；客户必须听自己的作品并修改。然而，有了Ultrax，拥有固态硬盘的人实际上可以看到自己舌头的运动，并利用这些信息来修改自己的语言。通过在下巴下放置一个标准的医用超声探头，捕捉舌头的运动已经成为可能。超声波有可能提供有关非典型语音的有力信息，并使说话者能够纠正自己不正确的发音。然而，图像是颗粒状的，信息（特别是关于舌尖的信息）经常丢失，图像很难解释。我们将通过利用关于舌头形状和运动范围的先验知识来改进这一图像，以便在超声图像序列中跟踪舌头轮廓提供有价值的约束。我们将应用一个舌模型来解决这个问题，利用显式的基于序列的优化来进行动态跟踪和平滑。我们将使用这项技术来增强超声波图像，将它们转化为动态的，我们假设这将A)儿童更容易理解B)将可见舌形的范围从只有元音和/r/扩展到包括/t/，/k/，/ch/和其他经常作为治疗目标的辅音。Ultrax将用于为ssd患者提供生物反馈治疗，并通过比较治疗前后的舌形提供客观评估进展的手段。我们将收集12名成人（超声和MRI）和90名小学生（超声）舌头运动的超声和MRI图像的大型数据库，以此为基础建立舌头轮廓模型并测试其性能。同时，我们将90名儿童分成3组，记录每组儿童对3种超声显示方式之一的反应：1。原始的，未增强的，超声波。附加解剖背景（如牙齿和上颚的位置）的非增强超声全增强超声，在这个项目中开发。儿童模仿舌头形状和动作的能力将被评估，以确定他们是否发现增强图像比未增强图像更容易理解，从而提高使用超声波进行生物视觉反馈的能力。我们将对9名患有ssd的儿童（每种类型3名儿童）进行超声治疗试验，使我们能够评估治疗过程中出现的实际问题，并为未来的临床试验铺平道路。在我们的研究项目结束时，我们将开发一种新的视觉反馈工具（Ultrax）的基础，供言语和语言治疗师用于ssd的诊断和治疗。

项目成果

Glottal Spectral Separation for Speech Synthesis

用于语音合成的声门频谱分离

• DOI: 10.1109/jstsp.2014.2307274
• 发表时间: 2014
• 期刊: IEEE Journal of Selected Topics in Signal Processing
• 影响因子: 7.5
• 作者: Cabral J

HMM-based speech synthesiser using the LF-model of the glottal source

• DOI: 10.1109/icassp.2011.5947405
• 发表时间: 2011-05
• 期刊: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• 作者: João P. Cabral;S. Renals;J. Yamagishi;Korin Richmond

Mage - Reactive articulatory feature control of HMM-based parametric speech synthesis

Mage - 基于 HMM 的参数语音合成的反应发音特征控制

• 发表时间: 2013
• 作者: Astrinaki M

Mage-HMM-based speech synthesis reactively controlled by the articulators

由发音器反应控制的基于 Mage-HMM 的语音合成

• 发表时间: 2013
• 作者: Astrinaki M

Comparing articulatory images: An MRI / Ultrasound Tongue Image database

比较关节图像：MRI/超声舌头图像数据库

• 发表时间: 2011
• 作者: Cleland J