Digitizing Human Vocal Interaction to Understand and Diagnose Autism

数字化人类声音互动以理解和诊断自闭症

• 批准号: 10406974
• 负责人: Julia Parish-Morris
• 金额: $ 58.87万
• 依托单位: CHILDREN'S HOSP OF PHILADELPHIA
• 依托单位国家: 美国
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-06-01 至 2025-05-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10406974
• 关键词: Acoustics; Adolescent; Affect; Algorithms; Autism Diagnosis; Behavior; Caring; Clinical; Clinical Research; Clinical Treatment; Clinical Trials; Companions; Complex; Computational Linguistics; Computer Vision Systems; Consumption; Control Groups; Decision Making; Development; Diagnosis; Diagnostic; Dimensions; Disease; Face; Facial Expression; Frequencies; Funding; Furuncles; Future; Genetic study; Human; Individual; Individual Differences; Language; Left; Machine Learning; Measurement; Measures; Mental Health; Mental disorders; Methods; Modality; Modeling; Monitor; Motor; National Institute of Mental Health; National Institute on Deafness and Other Communication Disorders; Outcome Measure; Output; Participant; Persons; Phenotype; Pilot Projects; Public Health; Sampling; Signal Transduction; Social Behavior; Social Interaction; Specificity; Stream; Techniques; Teenagers; Testing; Time; autism spectrum disorder; automated speech recognition; clinical care; clinical outcome measures; cognitive process; data fusion; design; digital; dyadic interaction; effective therapy; genetic association; improved; individuals with autism spectrum disorder; machine learning classification; machine learning classifier; machine learning model; meetings; motor behavior; multimodal data; multimodality; natural language; novel; precision medicine; skills; social; social communication; social deficits; sound; tool; trait

Abstract This proposal tackles an urgent need for sensitive clinical outcome measures of autism spectrum disorder (ASD) by developing an objective, digital, multi-modal social communication metric using computational linguistics (e.g., acoustic features, turn-taking rates, word frequency metrics). Our automatic speech recognition and natural language analytics approach is designed to fix known weaknesses in traditional measurements by providing granular information in less time, with built-in scalability for characterizing very large samples. Since ASD is defined by observables, it is ripe for an automated approach to digitizing behavior (e.g., words, sounds, facial expressions, motor behaviors). This proposal piggybacks on a recently funded R01 that uses computer vision and machine learning to characterize nonverbal motor synchrony in teens with either ASD or another disorder in a brief social conversation (MH118327, PI: Schultz). Vocal components of the conversation are not studied in MH118327; thus, the richness of the verbal domain is left untapped. We hypothesize that automatically derived spoken language markers will significantly predict group and individual differences in social communication skill, and – when fused with nonverbal features – will lead to better prediction than either modality alone. Together, these two projects represent a rare chance to study all observable social signals emitted during social interaction in the same diverse sample of participants. If funded, this project will be the first to use short conversations and multi-modal data fusion to predict social communication skill and diagnostic group in a large, clinically diverse sample of individuals with ASD and other disorders. Our pilot studies showed that a relatively small set of vocal features from a six-minute interaction predicts diagnosis (ASD vs. typical development [TD]) with 84% accuracy. These machine learning analyses also predicted social communication skill dimensionally, providing a granular metric of individual differences. Combining this approach with nonverbal metrics (R01MH118327) using decision level data fusion resulted in significantly better ASD vs. TD prediction – 91% accuracy. These pilot results are promising, but several gaps remain. In Aim 1 of this proposal, we assess the specificity of our vocal social communication approach by including a non-ASD psychiatric control group in our machine learning classification models, in addition to ASD and TD groups (N=250/group). In Aim 2, we clinically validate our transdiagnostic dimensional metric in a large, diverse sample of participants. In Aim 3, we test whether novel, sophisticated multi-modal fusion methods that combine vocal and nonverbal social communication features result in improved individual and group prediction. This proposal lays critical groundwork for an automated, precision medicine approach to studying, diagnosing, and caring for individuals with ASD and other mental health conditions. Suc- cessful completion of this project will transform how we quantify human behavior for a broad array of applications that demand efficient, scalable, and reliable measurement (e.g., genetic association studies, clinical trials and standard clinical care), thus meeting multiple strategic priorities set by NIMH and NIDCD.

摘要 这项建议解决了对自闭症谱系障碍(Asd)敏感的临床结果测量的迫切需要。 通过使用计算语言学开发客观的、数字的、多模式的社交沟通指标 (例如，声学特征、话轮转换率、词频度量)。我们的自动语音识别和自然 语言分析方法旨在修复传统测量方法中的已知弱点，方法是提供 在更短的时间内获得细粒度信息，具有内置的可扩展性，可用于表征非常大的样本。由于ASD是 根据可观察到的定义，将行为(例如，单词、声音、面部)数字化的自动化方法已经成熟 表情、运动行为)。这项提议是在最近资助的使用计算机视觉的R01上提出的 和机器学习来表征患有自闭症或其他障碍的青少年的非语言运动同步性 在简短的社交对话中(MH118327，PI：Schultz)。对话中的发声部分未在 MH118327；因此，语言领域的丰富性没有得到开发。我们假设自动派生出 口语标记将显著预测群体和个人在社交技能上的差异， 而且，当与非语言特征融合时，将比单独使用任何一种方式都能产生更好的预测。一起， 这两个项目是研究社会互动过程中发出的所有可观察到的社会信号的难得机会 在同一个不同的参与者样本中。如果获得资金，这个项目将是第一个使用短对话和 多模式数据融合预测社会沟通技能和诊断群体中的大型、临床多样性 患有自闭症和其他疾病的个体样本。我们的初步研究表明，相对较小的一组声乐 来自六分钟互动的特征预测诊断(ASD与典型发展[TD])的准确率为84%。 这些机器学习分析还在维度上预测了社交技能，提供了细粒度的 衡量个体差异的指标。使用Decision将此方法与非语言指标(R01MH118327)相结合 水平数据融合导致ASD显著优于TD预测-91%的准确率。这些试点结果是 前景看好，但仍有几个差距。在本提案的目标1中，我们评估了我们的声音社会 将非自闭症精神控制组纳入我们的机器学习分类中的沟通方法 模型组，ASD组和TD组(N=250只/组)。在目标2中，我们在临床上验证了我们的跨诊断 在一个大的、不同的参与者样本中的维度度量。在目标3中，我们测试新的、复杂的 结合有声和非语言社交特征的多模式融合方法改进了 个人和群体预测。这项提议为自动化、精确的医学奠定了关键的基础 研究、诊断和照顾患有自闭症和其他精神健康问题的个人的方法。Suc- 这个项目的圆满完成将改变我们对人类行为进行量化的方式 需要高效、可扩展和可靠的测量的应用程序(例如，遗传关联研究、临床 试验和标准临床护理)，从而满足NIMH和NIDCD设定的多个战略优先事项。