Wearable silent speech technology to enhance impaired oral communication

可穿戴式无声语音技术可增强受损的口语交流

• 批准号: 10218134
• 负责人: Jun Wang
• 金额: $ 58.12万
• 依托单位: UNIVERSITY OF TEXAS AT AUSTIN
• 依托单位国家: 美国
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10218134
• 关键词: Acoustics; Age; Alaryngeal Speech; Algorithms; Articular Range of Motion; Articulation; Articulators; Characteristics; Communication; Data; Data Collection; Development; Devices; Electrolarynx; Electromagnetics; Enhancement Technology; Esophageal Speech; Excision; Gender; Goals; Gold; Health; Hoarseness; Human; Impairment; Individual; Knowledge; Laryngeal Prosthesis; Laryngectomee; Laryngectomy; Larynx; Life; Lip structure; Machine Learning; Magnetism; Malignant neoplasm of larynx; Maps; Measures; Mechanics; Mental Depression; Motion; Motor; Movement; Output; Participant; Patients; Pattern; Performance; Population; Positioning Attribute; Production; Quality of life; Research; Robotics; Science; Self-Help Devices; Speech; Speech Disorders; Speech Sound; Speech Synthesizers; Speech Therapy; Speed; Technology; Testing; Time; Tongue; Tracer; Tracheoesophageal Speech; United States; Voice; Voice Disorders; Voice Quality; Wireless Technology; alternative communication; auditory feedback; base; design; experimental study; improved; innovation; kinematics; language training; machine learning algorithm; millisecond; new technology; novel; oral communication; preservation; prototype; social; social exclusion; sound; speech synthesis; tool; vibration; visual feedback; vocal cord; wearable device

Project Summary/Abstract The long-term objectives of this project are to obtain a deeper understanding how articulatory movement patterns are mapped to speech particularly when there is no vocal fold vibration (silent speech) and then to develop a novel, wearable assistive technology called silent speech interface (SSI) to assist the impaired oral communication for individuals in need (e.g., individuals after laryngectomy, surgical removal of larynx to treat advanced laryngeal cancer). Designed for daily use, the SSI contains a wearable magnetic device and a small camera for tongue and lip motion tracking, respectively, and an articulation-to-speech synthesizer to output natural sounding speech that preserves the speaker’s voice characteristics. Specific Aims of the proposal include to (1) determine the articulatory patterns of normal (vocalized) and silent speech, produced by both healthy talkers and people after laryngectomy, (2) develop a wearable, wireless magnetic device for real-time tongue and lip motion tracking, and (3) synthesize speech from articulation directly. There are currently limited alternative communication options for people who have undergone laryngectomy. These options include esophageal speech, tracheo-esophageal speech, and use of an artificial larynx (or electrolarynx). These solutions are either invasive or difficult to use, and all of them result in a hoarse or mechanical/robotic sounding voice, which can be difficult to understand. In contrast, the SSI in this application is non-invasive, easy-to-use, and produces natural sounding speech and may even preserve the patient’s voice identity. We have exciting preliminary results that support the feasibility of the project including that (1) we have recently developed a wireless magnetic device for tongue motion, and (2) we have demonstrated real- time articulation-to-speech synthesis with a 90% word accuracy (judged by a human listener). In this project, we will further reduce the size of the wireless device and make it wearable and conduct articulation-to-speech algorithms by studying 30 participants after laryngectomy and 30 age- and gender-matched healthy controls. If successful, the proposed research will enhance human health by making an impact on individuals after laryngectomy and potentially to a broader range of other speech and voice disorders. In addition, the technology will have an impact to the speech science field by providing a fist-time-ever tool for potential large- scale tongue motion data collection and have a variety of broader implications including visual feedback-based secondary language training and speech therapy, which may benefit millions of people with motor speech deficits in the United States.

项目摘要/摘要 该项目的长期目标是获得更深入的了解如何发音运动 图案映射到语音，尤其是在没有人声折叠振动（无声语音）然后到 开发一种名为“无声语音界面（SSI）”的小说，可穿戴的辅助技术，以帮助口服受损 有需要的个体的交流（例如，喉切除术后的个体，手术去除喉部以治疗 晚期喉癌）。 SSI专为日常使用而设计 分别用于舌头和唇部运动跟踪的摄像头和输出的发音到语音合成器 自然发声的演讲可以保留说话者的语音特征。提案的具体目的 包括（1）确定正常（发声）和无声言语的发音模式，两者都会产生 喉切除术后健康的说话者和人，（2）开发一种可穿戴的无线磁性设备，以实时 舌头和唇部运动跟踪，以及（3）直接从发音中综合语音。 目前，对于接受喉切除术的人来说，替代沟通选择有限。 这些选项包括食道语音，气管食管语音以及人工喉的使用（或 electorarynx）。这些解决方案要么是侵入性的，要么难以使用，所有这些解决方案都会导致嘶哑或 机械/机器人的声音可能难以理解。相反，此应用中的SSI 是无创的，易于使用的，并且会产生自然的发声，甚至可以保留患者的 语音身份。我们有令人兴奋的初步结果，可以支持该项目的可行性，包括（1） 我们最近开发了一种用于舌运动的无线磁性设备，（2）我们已经证明 用90％的单词准确性（由人类听众判断）的时间表达式综合。在这个项目中， 我们将进一步降低无线设备的尺寸，使其可穿戴并进行发音到语音 通过研究喉切除术后30名参与者以及30年龄和性别匹配的健康对照组来研究算法。如果 成功的研究将通过对个人产生影响之后的个人来增强人类健康 喉切除术，并有可能进行更广泛的其他语音和语音疾病。另外， 技术将通过为潜在的大型工具提供拳头科学领域的影响 扩展舌头运动数据收集，并具有多种更广泛的含义，包括基于视觉反馈的 二级语言培训和语音疗法，这可能使数百万有运动言语的人受益 美国的缺陷。