EMG Voice Restoration

肌电图语音恢复

• 批准号: 10376786
• 负责人: Gianluca De Luca
• 金额: $ 58.08万
• 依托单位: ALTEC, INC.
• 依托单位国家: 美国
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-05-16 至 2023-09-30
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10376786
• 关键词: Acoustics; Affinity; Algorithm Design; Algorithms; Architecture; Articulators; Augmentative and Alternative Communication; Cancer Survivor; Clinical; Communication; Computer software; Computers; Cosmetics; Custom; Data; Devices; Discrimination; Electrolarynx; Engineering; Eye Movements; Face; Future; General Hospitals; Genetic Transcription; Gestures; Goals; Hand; Human; Impairment; Individual; Intuition; Laryngectomee; Laryngectomy; Life; Malignant neoplasm of larynx; Massachusetts; Measures; Mental Health; Methods; Modeling; Motor; Muscle; Neck; Operative Surgical Procedures; Oral cavity; Orthography; Participant; Patient Self-Report; Performance; Persons; Phase; Production; Protocols documentation; Provider; Quality of life; Series; Signal Transduction; Small Business Innovation Research Grant; Social Interaction; Socialization; Speech; Speech Disorders; Stress; Surface; System; Tablet Computer; Technology; Testing; Text; Time; Training; Transcript; Translations; Variant; Vision; Voice; Work; base; design; digital; feasibility testing; handheld mobile device; improved; indexing; lexical; malignant oropharynx neoplasm; neural network; phrases; prototype; restoration; sensor; showing emotion; social; speech recognition; success; time use; vocalization; wearable sensor technology

Nearly 7.5 million people live without the ability to vocalize effectively. Existing augmentative and alternative communication (AAC) technology provides some function for these individuals, typically by converting physical gestures, eye movements or text into words that can be acoustically synthesized or visually displayed. However, a key limitation of these devices is that they do not involve natural mechanisms of speech production and therefore can be less intuitive as substitutes for the human vocal system. Consequently, they can suffer from lexical ambiguity, lack of emotional expression, and difficulty in conveying intent. There remains an unmet need to restore the natural mechanisms of speech production for the vocally impaired. To meet this need, we propose to develop a first-of-its-kind AAC system that restores personalized, prosodic, near real-time vocalization based on surface electromyographic (sEMG) signals produced during subvocal (i.e., silently mouthed) speech. In Phase I, we demonstrated the ability to recognize orthographic content and categorize emphatic stress between phrases subvocalized by (n=4) control and (n=4) post-laryngectomy participants with a 96.3% word recognition rate and 91.2% emphatic stress discrimination rate, respectively. Subvocal speech corpus transcripts were synthesized into prosodic speech using personalized, digital voices unique to each participant, then evaluated by naïve listeners (n=12). Listeners consistently rated our sEMG-based digital voice as having greater intelligibility, acceptability, emphasis discriminability and vocal affinity than the state-of-the-art electrolarynx (EL) speech aid used by laryngectomees. Having achieved these capabilities with lengthy post-processing of single phrases, we now aim to advance this technology in Phase II by solving the more fundamental challenges of transcribing prosodic speech and tracking variations in intonation and timing in near-real-time to restore conversational interactions in everyday life. To achieve this goal, our team of engineers at Altec Inc. is partnering with the world’s leading provider of personalized digitized voice for AAC (VocaliD, Inc), and world-class laryngeal cancer clinical experts (Massachusetts General Hospital) to develop algorithms for transcribing prosodic speech and tracking variations in intonation and timing throughout narratives, monologues and conversations (Aim 1); design MyoVoice™ system for near real-time mobile use (Aim 2); and evaluate the prototype system for conversational efficacy (Aim 3). Our milestone is to demonstrate within-subject improvements in ease-of-use, functional efficacy, and social reception amongst post-laryngectomy participants using our sEMG-based digital voice when compared to their typical EL speech aid. The final deliverable will consist of a single 4-contact sensor veneer and cross-platform, near-real-time mobile software that can operate on an AAC tablet or mobile device. Once commercialized, our vision for the future of this device is for a person—who is facing the devastating need to undergo laryngectomy—to have their voice banked and subvocal models trained such that immediately following surgery, they can receive a custom MyoVoice™ system to restore their original voice.

近750万人生活在没有有效发声能力的生活中。现有的增值性和替代性 通信(AAC)技术为这些个人提供了一些功能，通常是通过将 物理手势、眼球运动或文本转换为可声学合成或视觉显示的文字。 然而，这些设备的一个关键限制是它们不涉及自然的语音产生机制 因此，作为人类发声系统的替代品，它可能不那么直观。因此，他们可能会遭受 词汇含糊，缺乏情感表达，难以传达意图。仍然有一个未得到满足的需求 为发声障碍者恢复自然的语音产生机制。为了满足这一需求，我们建议 开发首个基于个性化、韵律、近乎实时发声的AAC系统 在亚声(即，无声)讲话期间产生的表面肌电(SEMG)信号。同相 I，我们演示了识别正字法内容和对短语之间的重音进行分类的能力 对照组(n=4)和喉切除术后组(n=4)的单词识别率分别为96.3%和 强调应激区分率分别为91.2%。合成了副语音语料库的转录 使用每个参与者独有的个性化数字声音转换为韵律语音，然后由朴素进行评估 听者(n=12)。听众一致认为我们基于表面肌电信号的数字语音具有更高的清晰度， 与最先进的电喉(EL)语音助听器相比，可接受性、强调区分度和发声亲和力更高 被喉切除者使用。通过对单个短语进行长时间的后处理，我们实现了这些功能 现在的目标是通过解决转录的更基本的挑战，在第二阶段推进这项技术 近乎实时的韵律语音和跟踪语调和计时的变化，以恢复对话 日常生活中的互动。为了实现这一目标，我们在Altec Inc.的工程师团队正在与 全球领先的AAC(vocaliD，Inc.)个性化数字化语音提供商，以及世界级的喉部 癌症临床专家(马萨诸塞州总医院)开发转录韵律语音的算法 并跟踪叙述、独白和对话中语调和时间的变化(目标1)； 为接近实时的移动应用设计MyoVoice™系统(目标2)；并对原型系统进行评估 会话效能(目标3)。我们的里程碑是展示主题内部在易用性方面的改进， 我们基于表面肌电信号的数字喉切除术后参与者的功能疗效和社会接受度 与他们典型的EL语音助手相比，他们的声音。最终交付的产品将由一个4触点传感器组成 单板和跨平台、近乎实时的移动软件，可在AAC平板电脑或移动设备上运行。 一旦商业化，我们对这种设备的未来的愿景是为面临毁灭性需求的人而设计的 接受喉切除术--让他们的嗓音被储存起来，并训练亚声模型，以便立即 手术后，他们可以接受定制的MyoVoice™系统，以恢复他们原来的声音。

项目成果

Surface Electromyography-Based Recognition, Synthesis, and Perception of Prosodic Subvocal Speech.

• DOI: 10.1044/2021_jslhr-20-00257
• 发表时间: 2021-05
• 期刊: Journal of speech, language, and hearing research : JSLHR
• 作者: Jennifer M. Vojtech;Michael D. Chan;Bhawna Shiwani;Serge H. Roy;J. Heaton;Geoffrey S. Meltzner;Paola Contessa;G. De Luca;R. Patel;Joshua C. Kline

Prediction of Voice Fundamental Frequency and Intensity from Surface Electromyographic Signals of the Face and Neck.

• DOI: 10.3390/vibration5040041
• 发表时间: 2022-12
• 期刊: Vibration
• 影响因子: 2
• 作者: Vojtech JM;Mitchell CL;Raiff L;Kline JC;De Luca G
• 通讯作者: De Luca G