Wearable silent speech technology to enhance impaired oral communication

可穿戴式无声语音技术可增强受损的口语交流

• 批准号: 10456297
• 负责人: Jun Wang
• 金额: $ 56.34万
• 依托单位: UNIVERSITY OF TEXAS AT AUSTIN
• 依托单位国家: 美国
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10456297
• 关键词: Acoustics; Age; Alaryngeal Speech; Algorithms; Articular Range of Motion; Articulation; Articulators; Characteristics; Communication; Data; Data Collection; Development; Devices; Electrolarynx; Electromagnetics; Enhancement Technology; Esophageal Speech; Excision; Gender; Goals; Gold; Health; Hoarseness; Human; Impairment; Individual; Knowledge; Laryngeal Prosthesis; Laryngectomee; Laryngectomy; Larynx; Life; Lip structure; Machine Learning; Magnetism; Malignant neoplasm of larynx; Maps; Measures; Mechanics; Mental Depression; Motion; Motor; Movement; Output; Participant; Patients; Pattern; Performance; Persons; Population; Positioning Attribute; Production; Quality of life; Research; Robotics; Science; Self-Help Devices; Speech; Speech Disorders; Speech Sound; Speech Synthesizers; Speech Therapy; Speed; Technology; Testing; Time; Tongue; Tracer; Tracheoesophageal Speech; United States; Voice; Voice Disorders; Voice Quality; Wireless Technology; alternative communication; auditory feedback; base; design; experimental study; improved; innovation; kinematics; language training; machine learning algorithm; millisecond; new technology; novel; oral communication; preservation; prototype; social; social exclusion; sound; speech synthesis; tool; vibration; visual feedback; vocal cord; wearable device; wireless; wireless electronic

Project Summary/Abstract The long-term objectives of this project are to obtain a deeper understanding how articulatory movement patterns are mapped to speech particularly when there is no vocal fold vibration (silent speech) and then to develop a novel, wearable assistive technology called silent speech interface (SSI) to assist the impaired oral communication for individuals in need (e.g., individuals after laryngectomy, surgical removal of larynx to treat advanced laryngeal cancer). Designed for daily use, the SSI contains a wearable magnetic device and a small camera for tongue and lip motion tracking, respectively, and an articulation-to-speech synthesizer to output natural sounding speech that preserves the speaker’s voice characteristics. Specific Aims of the proposal include to (1) determine the articulatory patterns of normal (vocalized) and silent speech, produced by both healthy talkers and people after laryngectomy, (2) develop a wearable, wireless magnetic device for real-time tongue and lip motion tracking, and (3) synthesize speech from articulation directly. There are currently limited alternative communication options for people who have undergone laryngectomy. These options include esophageal speech, tracheo-esophageal speech, and use of an artificial larynx (or electrolarynx). These solutions are either invasive or difficult to use, and all of them result in a hoarse or mechanical/robotic sounding voice, which can be difficult to understand. In contrast, the SSI in this application is non-invasive, easy-to-use, and produces natural sounding speech and may even preserve the patient’s voice identity. We have exciting preliminary results that support the feasibility of the project including that (1) we have recently developed a wireless magnetic device for tongue motion, and (2) we have demonstrated real- time articulation-to-speech synthesis with a 90% word accuracy (judged by a human listener). In this project, we will further reduce the size of the wireless device and make it wearable and conduct articulation-to-speech algorithms by studying 30 participants after laryngectomy and 30 age- and gender-matched healthy controls. If successful, the proposed research will enhance human health by making an impact on individuals after laryngectomy and potentially to a broader range of other speech and voice disorders. In addition, the technology will have an impact to the speech science field by providing a fist-time-ever tool for potential large- scale tongue motion data collection and have a variety of broader implications including visual feedback-based secondary language training and speech therapy, which may benefit millions of people with motor speech deficits in the United States.

项目总结/摘要 本项目的长期目标是更深入地了解发音运动 模式被映射到语音，特别是当没有声带振动（无声语音）时，然后， 开发一种新的，可穿戴的辅助技术，称为无声语音接口（SSI），以帮助受损的口语 为有需要的个人提供通信（例如，个人喉切除术后，手术切除喉治疗 晚期喉癌）。SSI专为日常使用而设计，包含一个可穿戴的磁性设备和一个小的 分别用于舌头和嘴唇运动跟踪的摄像机，以及用于输出 保留说话者声音特征的自然发声语音。提案的具体目标 包括（1）确定正常（发声）和无声语音的发音模式，由两者产生 健康的谈话者和喉切除术后的人，（2）开发一种可穿戴的无线磁性设备， 舌唇运动跟踪;（3）直接从发音合成语音。 目前，对于接受过喉切除术的人来说，替代性交流选择有限。 这些选择包括食管发音、气管食管发音和使用人工喉（或 电子喉）。这些解决方案要么是侵入性的，要么是难以使用的，所有这些都导致嘶哑或 机械/机器人的声音，这可能很难理解。相比之下，本申请中的SSI 是非侵入性的，易于使用，并产生自然发声的语音，甚至可以保留患者的 语音识别我们有令人兴奋的初步结果，支持该项目的可行性，包括（1） 我们最近开发了一种用于舌头运动的无线磁装置，以及（2）我们已经证明了真实的- 时间发音到语音合成，具有90%的单词准确度（由人类听众判断）。在本项目中， 我们将进一步缩小无线设备的尺寸，使其可穿戴，并进行发音到语音 通过研究30名喉切除术后的参与者和30名年龄和性别匹配的健康对照者，如果 如果成功，拟议的研究将通过对个人产生影响来增强人类健康。 喉切除术和潜在的更广泛的其他语言和声音障碍。此外该 技术将对语音科学领域产生影响，为潜在的大型 缩放舌头运动数据收集并具有各种更广泛的含义，包括基于视觉反馈的 第二语言训练和言语治疗，这可能会使数百万运动言语患者受益 美国的赤字。