The automated coding of expressed emotion to enhance clinical and epidemiological mental health research in adolescence

表达情绪的自动编码，以加强青春期的临床和流行病学心理健康研究

• 批准号: MR/X002721/1
• 负责人: Johnny Downs
• 金额: $ 38.69万
• 依托单位: King's College London
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2022
• 资助国家: 英国
• 起止时间: 2022 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=MR%2FX002721%2F1
• 关键词: automated; coding; expressed; emotion; enhance

As little as five minutes listening to a parent talk can reveal a significant amount of information about their child's future psychopathology. Analyses of the words and tone used in parents' speech, provides a detailed picture about the parents, the child, and interactions within the family. An interview technique called the five-minute speech sample (FMSS) has operationalised this process. There is good evidence the FMSS can provide an index of a child's home environment and help profile their risk of developing, and recovering from, adolescent-onset mental health disorders. FMSS are easy to collect - all you need is 5 minutes and a dictaphone/smartphone - yet they are rarely used in research or clinical settings, because the coding of speech is laborious, bias-prone, and requires highly trained raters. If these issues could be overcome, FMSS presents tremendous opportunity to be used across research, mental health and social care settings to rapidly assess key modifiable drivers of mental health problems among adolescents.This project will bring together an interdisciplinary team of developmental psychopathologists, creative writers, plus computer, clinical and social scientists to automate the coding of the FMSS. Using recent developments in computational linguistics and affective computing, we will create a pipeline which combines automatic translation, speech valance and natural language analysis. We will use a unique collection of researcher-rated FMSS audio recordings of mothers from the UK E-Risk Longitudinal Twin Study, which were obtained on 2031 children at 10 years of age. The children in this cohort have been followed to age 18 years, undergoing multiple waves of comprehensive assessments. Building on an existing feasibility study, we will develop and train an automated approach to FMSS coding - exploiting both the size and socio-economic representativeness of the sample. We will then examine whether the automated ratings generated from age-10 maternal speech samples show the same ability to predict mental health problems at 12 and 18 years as well as costly human ratings. We will conduct creative workshops with key stakeholders (young people, parents, healthcare/social-work practitioners, research policymakers, governance leads, and educators) throughout the project to determine the main ethical, social and practical challenges to using and sharing parental speech data and developing and implementing the automated models in practice to inform future work. Crucially, all this work will help us understand how to share this methodology, and as a final output we will develop open-source materials and a clear blueprint of what is required to build a secure digital platform that enables other research groups to rapidly code expression emotion from FMSS in an accurate and cost-effective manner.

只要听父母五分钟的谈话，就能揭示出孩子未来精神病理的大量信息。通过对父母讲话中使用的词语和语气的分析，可以详细了解父母、孩子以及家庭中的互动。一种名为“五分钟语音样本”（FMSS）的面试技巧将这一过程付诸实践。有很好的证据表明，FMSS可以提供一个儿童家庭环境的指数，并帮助描述他们发展的风险，并从青春期开始的精神健康障碍中恢复。FMSS很容易收集——你所需要的只是5分钟和一部录音机/智能手机——但它们很少用于研究或临床环境，因为语音编码很费力，容易产生偏见，而且需要训练有素的评分员。如果能够克服这些问题，FMSS将提供巨大的机会，用于研究、心理健康和社会保健机构，以快速评估青少年心理健康问题的主要可改变驱动因素。这个项目将汇集一个由发展精神病理学家、创意作家、计算机、临床和社会科学家组成的跨学科团队，使FMSS的编码自动化。利用计算语言学和情感计算的最新发展，我们将创建一个结合自动翻译，语音价值和自然语言分析的管道。我们将使用来自英国E-Risk纵向双胞胎研究的研究者评级的FMSS母亲录音的独特收集，这些录音来自2031名10岁的儿童。该队列中的儿童一直被跟踪到18岁，经历了多波综合评估。在现有可行性研究的基础上，我们将开发和培训一种自动化的FMSS编码方法——利用样本的规模和社会经济代表性。然后，我们将检验从10岁母亲语言样本中生成的自动评分是否显示出与人类评分相同的预测12岁和18岁心理健康问题的能力。我们将在整个项目中与主要利益相关者（年轻人、父母、医疗保健/社会工作从业人员、研究决策者、治理领导和教育工作者）举办创意研讨会，以确定使用和共享父母语音数据以及在实践中开发和实施自动化模型以为未来工作提供信息的主要道德、社会和实践挑战。至关重要的是，所有这些工作将帮助我们了解如何分享这种方法，作为最终的成果，我们将开发开源材料和一个清晰的蓝图，以建立一个安全的数字平台，使其他研究小组能够以准确和经济有效的方式快速编码来自FMSS的表达情感。

项目成果

Automatic Detection of Expressed Emotion from Five-Minute Speech Samples: Challenges and Opportunities

自动检测五分钟语音样本中表达的情绪：挑战和机遇

• DOI: 10.21437/interspeech.2022-10188
• 发表时间: 2022
• 作者: Mirheidari B

Towards robust paralinguistic assessment for real-world mobile health (mHealth) monitoring: an initial study of reverberation effects on speech

对现实世界移动健康 (mHealth) 监测进行稳健的副语言评估：语音混响效应的初步研究

• DOI: 10.21437/interspeech.2023-947
• 发表时间: 2023
• 作者: Dineley J