Artificial Intelligence Applied to Video and Speech for Objectively Evaluating Social Interaction and Depression in Mild Cognitive Impairment

项目总结/摘要 据报道，社会参与度的下降以及抑郁和孤独感的减少， 与轻度认知障碍（MCI）的进展相关。然而，以前的研究主要依赖于 在回顾性调查分析中，缺乏对真实的行为生物标志物的连续量化 世界人工智能（AI）的最新进展使其能够评估人类的身体和精神 在真实的世界中的活动。然而，这些技术主要是在正常人群中进行评估， 与MCI个体明显不同。该提案旨在验证应用于视频的AI技术 和言语来客观地评估MCI人群中的社会交往和心理健康。成功 这项提案的完成将为我们实现长期目标迈出重要一步，这是一个大规模的 在真实的世界中对MCI进展进行连续、客观量化的纵向研究。我们的一般 假设是，在现实世界中，社会参与和心理健康的微妙而重要的变化 MCI个体的相互作用可以被连续地捕获并量化以用于认知损害。为我们 在这项研究中，MCI患者将从佐治亚州亚特兰大市招募具有平衡种族和民族背景的患者， 区在目标1中，将在埃默里大学的认知授权项目（CEP）中进行社会参与分析 Goizueta阿尔茨海默病研究中心。CEP空间安装了一个基于边缘计算的隐私- 具有包括照相机的多模态传感器的保存、低成本的患者监测系统。在CEP， 招募的MCI患者将参加由治疗机构提供的身体和认知训练计划。 服务专业人员。多摄像机网络系统检测MCI患者的社会参与特征 将被用来预测领悟社会支持量表（PSS）和蒙特利尔认知评估（莫卡） 对应患者的评分。在目标2和3中，通过视频进行抑郁和孤独分析 来自招募的MCI患者的认知评估访谈（CAI）的记录。为金标准 抑郁和孤独量表，MCI参与者将提供老年抑郁症（GDS）和加州大学洛杉矶分校孤独症 的规模，分别。根据录像，MCI患者的面部（目标2）和言语（目标3）行为 将分析GDS、UCLA Longland和莫卡量表的预测。技术的验证 这项建议预计将对量化MCI进展产生重大影响，因为这些技术可以 很容易扩展到量化真实的世界中的其他MCI相关行为，如徘徊或冷漠。还有， 该提案在社会和心理健康特征方面的发现可以推动新的假设产生 用于开发认知能力下降的新疗法的临床试验。这项研究符合NIA的使命， 了解与衰老过程相关的疾病和状况，以延长健康的寿命 通过支持对老龄化的行为研究和向科学界传播研究信息。

Project Summary/Abstract The decreases in social engagement and the feeling of depression and loneliness are reported to be highly correlated with the progression of mild cognitive impairment (MCI). However, previous studies primarily relied on retrospective survey analysis, which lacks continuously quantifying such behavioral biomarkers in the real world. Recent advances in artificial intelligence (AI) made it capable of assessing human physical and mental activities in the real world. Yet, those techniques are mainly evaluated on normal populations, whose behaviors are distinctively different from MCI individuals. This proposal aims to validate AI technologies applied to video and speech for objectively evaluating social interaction and mental health in a MCI population. Successful completion of this proposal will provide an important step toward our long-term goal, which is a large-scale longitudinal study for continuous, objective quantification of MCI progression in the real world. Our general hypothesis is that the subtle yet important changes in social engagement and mental health during real-world interactions of MCI individuals can be continuously captured and quantified for cognitive impairment. For our study, MCI patients will be recruited with balanced race and ethnic backgrounds from the urban Atlanta, GA, area. In Aim 1, social engagement analysis will be conducted at Cognitive Empowerment Program (CEP) at Emory Goizueta Alzheimer’s Disease Research Center. The CEP space is installed with an edge computing-based, privacy- preserving, low-cost patient monitoring system having multi-modal sensors including cameras. At CEP, the recruited MCI patients will be participating in physical and cognitive training programs provided by therapeutic service professionals. MCI patients' social engagement features detected from the multi-camera network system will be used to predict the Perceived Social Support Scale (PSS) and Montreal Cognitive Assessment (MoCA) scores of corresponding patients. In Aims 2 and 3, depression and loneliness analysis is conducted with video recordings of the Cognitive Assessment Interview (CAI) from the recruited MCI patients. For gold standard depression and loneliness scales, MCI participants will provide Geriatric Depression (GDS) and UCLA Loneliness scales, respectively. From the video recordings, the facial (Aim 2) and speech (Aim 3) behavior of MCI patients will be analyzed for the prediction of GDS, UCLA Loneliness, and MoCA scales. The validation of techniques in this proposal is expected to have a significant impact on quantifying MCI progressions as these techniques can be readily extended to quantifying other MCI-related behaviors in the real world, such as wandering or apathy. Also, this proposal’s findings in social and mental health features can drive novel hypothesis generation to power clinical trials for developing novel treatments for cognitive decline. This research aligns with the NIA’s mission to understand the diseases and conditions associated with the aging process in order to extend healthy years of life by supporting behavioral research on aging and disseminating research information to the scientific community.