Wearable silent speech technology to enhance impaired oral communication

点击翻译按钮获取中文摘要

Project Summary/Abstract The long-term objectives of this project are to obtain a deeper understanding how articulatory movement patterns are mapped to speech particularly when there is no vocal fold vibration (silent speech) and then to develop a novel, wearable assistive technology called silent speech interface (SSI) to assist the impaired oral communication for individuals in need (e.g., individuals after laryngectomy, surgical removal of larynx to treat advanced laryngeal cancer). Designed for daily use, the SSI contains a wearable magnetic device and a small camera for tongue and lip motion tracking, respectively, and an articulation-to-speech synthesizer to output natural sounding speech that preserves the speaker’s voice characteristics. Specific Aims of the proposal include to (1) determine the articulatory patterns of normal (vocalized) and silent speech, produced by both healthy talkers and people after laryngectomy, (2) develop a wearable, wireless magnetic device for real-time tongue and lip motion tracking, and (3) synthesize speech from articulation directly. There are currently limited alternative communication options for people who have undergone laryngectomy. These options include esophageal speech, tracheo-esophageal speech, and use of an artificial larynx (or electrolarynx). These solutions are either invasive or difficult to use, and all of them result in a hoarse or mechanical/robotic sounding voice, which can be difficult to understand. In contrast, the SSI in this application is non-invasive, easy-to-use, and produces natural sounding speech and may even preserve the patient’s voice identity. We have exciting preliminary results that support the feasibility of the project including that (1) we have recently developed a wireless magnetic device for tongue motion, and (2) we have demonstrated real- time articulation-to-speech synthesis with a 90% word accuracy (judged by a human listener). In this project, we will further reduce the size of the wireless device and make it wearable and conduct articulation-to-speech algorithms by studying 30 participants after laryngectomy and 30 age- and gender-matched healthy controls. If successful, the proposed research will enhance human health by making an impact on individuals after laryngectomy and potentially to a broader range of other speech and voice disorders. In addition, the technology will have an impact to the speech science field by providing a fist-time-ever tool for potential large- scale tongue motion data collection and have a variety of broader implications including visual feedback-based secondary language training and speech therapy, which may benefit millions of people with motor speech deficits in the United States.