A Specialized Automatic Speech Recognition and Conversational Platform to Enable Socially Assistive Robots for Persons with Mild-to-Moderate Alzheimer's Disease and Related Dementia

摘要 美国三分之一的老年人死于痴呆症，其中阿尔茨海默病(AD)是最常见的 形式。AD患者的有意义的交流和互动能力降低，这导致 对专业照顾者和家庭成员来说都是巨大的压力和负担。社交辅助机器人 (SARS)旨在促进治疗互动和交流。不幸的是，人工的 智能(AI)长期以来一直受到老年人言语的挑战，老年人表现出与年龄相关的声音 震颤、犹豫不决、辅音发音不准确、基频变异性增加，以及 与阿尔茨海默病相关的神经变化可能加剧的其他障碍，通过 常见的环境噪音，如吊扇、电视等，因为由此产生的真实世界很差 通过现有的SAR技术理解语音和语言，稀缺的人类照顾者往往 需要通过SAR交互来指导AD患者，将SARS限制为小型部署，主要作为 研究性研究。与现有的单纯依赖人工智能的方法不同，Care.Coach™正在开发一种类似于SAR的方法 通过真正自然的语言与老年人和AD患者交谈的阿凡达。每个化身都由一个 由训练有素的员工组成的24x7全天候团队，他们可以经济高效地监控和接触12名或更多患者 顺序地(2同时)通过来自患者的虚拟化身设备的音频/视频馈送。工作人员 通过发送文本命令与每个患者交流，这些命令通过 一种语音合成引擎。工作人员将他们的语言能力和自然能力贡献给系统 语言处理(NLP)，并用于生成自由格式的对话响应，以帮助患者 与化身的私人关系。工作人员由嵌入到 他们的工作界面，通过基于证据的提示和协议进行编程，以支持健康 行为和自我照顾。该SBIR快速通道项目将利用由我们的人类- 环内平台，以开发新的ASR功能，支持全自动对话协议以 参与和支持AD患者，而无需人工干预。我们在第一阶段的目标是利用我们独特的 工作数据集，以训练自动语音识别(ASR)引擎，以使能够理解某些 老年人和AD患者的语音类型比目前任何可用的引擎都要成功。我们的目标是 第二阶段，将这个新引擎和一个NLP模块整合到我们现有的人在环中的化身中 系统，招募AD患者群体，在为期2年的人类受试者中进行进一步培训和验证 研究，以便我们可以演示我们的化身对话的很大一部分完全自动化- 中中度AD患者。因此，我们将提高我们化身的商业可伸缩性，同时验证 我们的新ASR/NLP引擎是支持下一代以AD为重点的SARS的最准确平台。

Abstract 1 in 3 seniors in the United States dies with dementia, of which Alzheimer’s disease (AD) is the most common form. AD patients suffer from decreased ability to meaningfully communicate and interact, which causes significant stress and burden for both professional caregivers and family members. Socially assistive robots (SARs) have been designed to promote therapeutic interaction and communication. Unfortunately, artificial intelligence (AI) has long been challenged by the speech of elderly persons, who exhibit age-related voice tremors, hesitations, imprecise production of consonants, increased variability of fundamental frequency, and other barriers that can be exacerbated by the neurological changes associated with AD, further complicated by common environmental noises such as the ceiling fan, television, etc. Because of the resulting poor real-world speech and language understanding by available SAR technologies, scarce human caregivers are often required to guide AD patients through SAR interactions, limiting SARs to small deployments, mostly as part of research studies. Unlike existing approaches relying purely on AI, care.coach™ is developing a SAR-like avatar that converses with elderly and AD patients through truly natural speech. Each avatar is controlled by a 24x7 team of trained human staff who can cost-effectively monitor and engage 12 or more patients sequentially (2 simultaneously) through the audio/visual feeds from the patient’s avatar device. The staff communicate with each patient by sending text commands which are converted into the avatar’s voice through a speech synthesis engine. The staff contribute to the system their human abilities for speech and natural language processing (NLP) and for generating free-form conversational responses to help patients build personal relationships with the avatar. The staff are guided by a software-driven expert system embedded into their work interface, which is programmed with evidence-based prompting and protocols to support healthy behaviors and self-care. This SBIR Fast-Track project will leverage the unique data generated by our human- in-the-loop platform to develop new ASR capabilities, enabling fully automatic conversational protocols to engage and support AD patients without human intervention. We aim in Phase I to leverage our unique prior work dataset to train an automatic speech recognition (ASR) engine to enable the understanding of certain types of elderly and AD patient speech more successfully than any currently available engine. We aim in Phase II to incorporate this new engine along with an NLP module into our existing human-in-the-loop avatar system, recruiting a population of AD patients to further train and validate with during a 2-year human subjects study so that we can demonstrate full automation of a significant portion of our avatar conversations with mild- to-moderate level AD patients. Thus, we will improve the commercial scalability of our avatars, while validating our new ASR/NLP engine as the most accurate platform for enabling the next generation of AD-focused SARs.