EMG Voice Restoration

肌电图语音恢复

• 批准号: 10009728
• 负责人: Gianluca De Luca
• 金额: $ 50.63万
• 依托单位: ALTEC, INC.
• 依托单位国家: 美国
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-05-16 至 2023-03-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10009728
• 关键词: Acoustics; Affinity; Algorithm Design; Algorithms; Architecture; Articulators; Augmentative and Alternative Communication; Cancer Survivor; Clinical; Communication; Computer software; Computers; Cosmetics; Custom; Data; Devices; Discrimination; Electrolarynx; Engineering; Eye Movements; Face; Future; General Hospitals; Genetic Transcription; Gestures; Goals; Hand; Human; Impairment; Individual; Intuition; Laryngectomee; Laryngectomy; Life; Malignant neoplasm of larynx; Massachusetts; Measures; Mental Health; Methods; Modeling; Motor; Muscle; Neck; Operative Surgical Procedures; Oral cavity; Orthography; Participant; Patient Self-Report; Performance; Persons; Phase; Production; Protocols documentation; Provider; Quality of life; Series; Signal Transduction; Small Business Innovation Research Grant; Social Interaction; Socialization; Speech; Speech Disorders; Stress; Surface; System; Tablet Computer; Technology; Testing; Text; Time; Training; Transcript; Translations; Variant; Vision; Voice; Work; base; design; digital; handheld mobile device; improved; indexing; lexical; malignant oropharynx neoplasm; neural network; phrases; prototype; restoration; sensor; showing emotion; social; speech recognition; success; time use; vocalization; wearable sensor technology

Nearly 7.5 million people live without the ability to vocalize effectively. Existing augmentative and alternative communication (AAC) technology provides some function for these individuals, typically by converting physical gestures, eye movements or text into words that can be acoustically synthesized or visually displayed. However, a key limitation of these devices is that they do not involve natural mechanisms of speech production and therefore can be less intuitive as substitutes for the human vocal system. Consequently, they can suffer from lexical ambiguity, lack of emotional expression, and difficulty in conveying intent. There remains an unmet need to restore the natural mechanisms of speech production for the vocally impaired. To meet this need, we propose to develop a first-of-its-kind AAC system that restores personalized, prosodic, near real-time vocalization based on surface electromyographic (sEMG) signals produced during subvocal (i.e., silently mouthed) speech. In Phase I, we demonstrated the ability to recognize orthographic content and categorize emphatic stress between phrases subvocalized by (n=4) control and (n=4) post-laryngectomy participants with a 96.3% word recognition rate and 91.2% emphatic stress discrimination rate, respectively. Subvocal speech corpus transcripts were synthesized into prosodic speech using personalized, digital voices unique to each participant, then evaluated by naïve listeners (n=12). Listeners consistently rated our sEMG-based digital voice as having greater intelligibility, acceptability, emphasis discriminability and vocal affinity than the state-of-the-art electrolarynx (EL) speech aid used by laryngectomees. Having achieved these capabilities with lengthy post-processing of single phrases, we now aim to advance this technology in Phase II by solving the more fundamental challenges of transcribing prosodic speech and tracking variations in intonation and timing in near-real-time to restore conversational interactions in everyday life. To achieve this goal, our team of engineers at Altec Inc. is partnering with the world’s leading provider of personalized digitized voice for AAC (VocaliD, Inc), and world-class laryngeal cancer clinical experts (Massachusetts General Hospital) to develop algorithms for transcribing prosodic speech and tracking variations in intonation and timing throughout narratives, monologues and conversations (Aim 1); design MyoVoice™ system for near real-time mobile use (Aim 2); and evaluate the prototype system for conversational efficacy (Aim 3). Our milestone is to demonstrate within-subject improvements in ease-of-use, functional efficacy, and social reception amongst post-laryngectomy participants using our sEMG-based digital voice when compared to their typical EL speech aid. The final deliverable will consist of a single 4-contact sensor veneer and cross-platform, near-real-time mobile software that can operate on an AAC tablet or mobile device. Once commercialized, our vision for the future of this device is for a person—who is facing the devastating need to undergo laryngectomy—to have their voice banked and subvocal models trained such that immediately following surgery, they can receive a custom MyoVoice™ system to restore their original voice.

将近750万人生活，没有能力有效发声。现有的增强性和替代方案 通信（AAC）技术为这些人提供了一些功能，通常通过转换 物理手势，眼动或文本成像可以合成或视觉上显示的单词。 但是，这些设备的关键限制是它们不涉及语音生产的自然机制 因此，作为人声系统的替代品可能不太直观。因此，他们可能遭受 词汇歧义，缺乏情感表达，并且在传达意图方面很难。仍然有未满足的需求 恢复声音受损的语音产生的自然机制。为了满足这种需求，我们提出 开发首个AAC系统，该系统恢复个性化的，韵律的，接近实时发声 在亚次肌电肌电图（SEMG）信号上，在次频（即默默嘴巴）语音中产生的信号。在阶段 i，我们证明了能够识别拼字法内容并分类短语之间强调的能力 通过（n = 4）对照和（n = 4）诊断后参与者的子评估，单词识别率为96.3％ 91.2％强调应力歧视率。综合了副语音语料库成绩单 使用每个参与独特的个性化的数字声音进入韵律语音，然后由幼稚评估 听众（n = 12）。听众始终将我们的基于SEMG的数字语音评为更高的清晰度， 可接受性，强调可区分性和声音亲和力比最先进的exterolarynx（EL）语音辅助 由喉头瘤使用。通过冗长的单词后处理，我们实现了这些功能，我们 现在，通过解决抄写更根本的挑战，以在第二阶段推进这项技术 韵律的语音和跟踪在近实时的语调和时间上的变化，以恢复对话 日常生活中的互动。为了实现这一目标，我们的Altec Inc.工程师团队正在与 全球领先的AAC个性化数字化声音提供商（Vocalid，Inc）和世界一流的喉部 癌症临床专家（马萨诸塞州总医院）开发用于转录韵律语音的算法 并在整个叙事，独白和对话中跟踪语调和时间的变化（AIM 1）； 设计用于接近实时移动使用的Myovoice™系统（AIM 2）；并评估原型系统 会话效率（AIM 3）。我们的里程碑是证明易用性的受试者内部改进， 使用我们的基于SEMG的数字 与典型的EL语音辅助相比，声音。最终输送将由一个4接触传感器组成 单板和跨平台，近实时的移动软件，可以在AAC平板电脑或移动设备上运行。 一旦商业化，我们对该设备未来的愿景就是一个人面临毁灭性需求的人 要进行喉切除术，以训练他们的声音和次数训练，以便立即 手术后，他们可以收到一个自定义的Myovoice™系统来恢复原始声音。