Expanding articulatory information from ultrasound imaging of speech using MRI-based image simulations and audio measurements

使用基于 MRI 的图像模拟和音频测量来扩展语音超声成像的发音信息

• 批准号: 10537976
• 负责人: Sarah Rotong Li
• 金额: $ 3.97万
• 依托单位: UNIVERSITY OF CINCINNATI
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-08-01 至 2024-07-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10537976
• 关键词: 3-Dimensional; Acoustics; Address; Adult; Affect; Age; Air; Area; Articulation; Biofeedback Training; Black race; Characteristics; Child; Client; Clinical; Computer software; Data; Data Collection; Data Set; Disease; Education; Employment; Feedback; Fellowship; Future; Geometry; Goals; Guidelines; Hand; Health; Image; Image Analysis; Individual; Institution; Interpersonal Relations; Intervention; Knowledge; Language; Language Development; Learning; Magnetic Resonance Imaging; Measurement; Methods; Modeling; Morphologic artifacts; Movement; Neural Network Simulation; Obstructive Sleep Apnea; Oryctolagus cuniculus; Outcome; Palate; Pathologist; Production; Recommendation; Research; Research Personnel; Rotation; Schools; Shapes; Source; Speech; Speech Disorders; Speech Sound; Speech Therapy; Structure; Stuttering; Surface; Techniques; Technology; Testing; Time; Tissue Model; Tissue imaging; Tissues; Tongue; Tooth structure; Training; Translational Research; Ultrasonic wave; Ultrasonography; United States; Universities; Variant; Visit; Visual; Work; base; deep learning; deep learning model; imaging Segmentation; imaging system; improved; insight; machine learning model; neural network; novel; novel strategies; predictive modeling; prevent; simulation; skills; social; sound; speech accuracy; success; tissue mapping; tongue apex; tool; ultrasound

PROJECT SUMMARY Ultrasound imaging provides articulatory feedback useful for remediating speech sound disorders, which affect 5% of children and cause long-term deficits in social health and employment in adulthood. However, ultrasound imaging can be difficult to interpret for clinicians and individuals, limiting the understanding of articulatory data and ultrasound biofeedback therapy speech outcomes. A likely source of difficulty is the articulatory information missing from ultrasound images, such as the tongue tip and reference vocal tract structures (e.g., palate) that cannot be consistently imaged with ultrasound due to air. Much of this missing information from ultrasound can be ascertained in magnetic resonance imaging (MRI) because MRI images the entire vocal tract. Comparing ultrasound images and MRI will improve interpretation of ultrasound images by confirming that certain characteristics of ultrasound images (e.g., obscured tongue tip, double edge artifacts) occur from characteristics of tongue shapes; as well, models can be trained to predict from ultrasound images the articulatory information shown in MRI. However, articulatory variability prevents direct comparison between these images. A novel approach to avoid variability is to simulate ultrasound wave propagation in tissue segmented from MRI. Recent advancements in deep learning have also demonstrated ability to address the inverse problem of predicting articulation from acoustic data. Thus, to meet the needs of improving ultrasound image interpretation, the goal for this proposal is to use simulated ultrasound images and neural network models to characterize and predict articulatory information missing from 2D midsagittal ultrasound images. These models will be trained on MRI and audio data. We will characterize missing articulatory information by developing efficient simulation of ultrasound images from MRI tissue segmentation. One hypothesis that will be tested is the guideline for using the lower edge of double edge artifacts in ultrasound images as the tongue surface. To test this guideline for a greater range of data (including disordered child speakers and different simulated probe rotations), double edge artifacts will be compared with tissue maps used to generate the simulated images. Another comparison will estimate the amount of tongue tip typically missing in /r/ tongue shapes. We will then develop a deep learning model that trains on information from MRI to predict midsagittal vocal tract shapes (including the tongue tip and palate) from the inputs of tongue contours from ultrasound and audio. With these aims, we will add insight to ultrasound imaging for speech and provide a tool with future applications in expanding articulatory information, e.g., testing outcomes of using more complete vocal tract information in ultrasound biofeedback therapy. Training for this fellowship will occur at the University of Cincinnati, with opportunities to visit labs at two additional institutions. The proposed plan provides training from a range of investigators in topics such as ultrasound imaging and application to speech research, developing skills needed for my future goals.

项目摘要 超声成像提供了发音反馈，可用于修复语音疾病， 影响5％的儿童，并在成年后造成社会健康和就业的长期赤字。 但是，对于临床医生和个人，超声成像可能很难解释，从而限制了 了解发音数据和超声生物反馈疗法的演讲结果。可能的来源 难度是超声图像中缺少的关节信息，例如舌头和参考 声带结构（例如，口感）无法通过空气而超声始终成像。 在磁共振成像中可以确定来自超声的许多缺失信息 （MRI）因为MRI图像整个声道。比较超声图像和MRI将改善 通过确认超声图像的某些特征（例如， 舌头遮盖的尖端，双边伪影）来自舌形的特征；同样，模型可以 接受培训以从超声图像中预测MRI中显示的发音信息。但是，发音 可变性阻止了这些图像之间的直接比较。避免可变性的一种新颖方法是 模拟从MRI分割的组织中的超声波传播。深度学习的最新进展 还表明了解决声学数据表达的反向问题的能力。 因此，为了满足改善超声图像解释的需求，该建议的目标是使用 模拟超声图像和神经网络模型，以表征和预测关节信息 2d中幕超声图像缺失。这些模型将接受MRI和音频数据的培训。 我们将通过开发有效的超声模拟来表征缺失的关节信息 来自MRI组织分割的图像。将测试的一个假设是使用较低的指南 超声图像中双边伪影的边缘作为舌表面。测试该指南的更大 数据范围（包括无序的儿童扬声器和不同的模拟探针旋转），双边缘 将伪影与用于生成模拟图像的组织图进行比较。另一个比较会 估计舌头尖端通常缺少 / r /舌头形状。然后，我们将发展深入学习 训练来自MRI的信息以预测中标的声音形状的模型（包括舌头和 口感）来自超声和音频的舌轮廓输入。以这些目标，我们将为 超声成像进行语音成像，并为扩大发音信息的未来应用提供工具， 例如，在超声生物反馈疗法中使用更完整的声带信息的测试结果。 该奖学金的培训将在辛辛那提大学举行，并有机会参观两个 其他机构。拟议的计划提供了一系列调查人员的培训，例如 超声成像和对语音研究的应用，发展我未来目标所需的技能。