Digitizing Human Vocal Interaction to Understand and Diagnose Autism

数字化人类声音互动以理解和诊断自闭症

• 批准号: 10631926
• 负责人: Julia Parish-Morris
• 金额: $ 68.08万
• 依托单位: CHILDREN'S HOSP OF PHILADELPHIA
• 依托单位国家: 美国
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-06-01 至 2025-05-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10631926
• 关键词: Acoustics; Adolescent; Affect; Algorithms; Autism Diagnosis; Behavior; Caring; Classification; Clinical; Clinical Treatment; Clinical Trials; Companions; Complex; Computational Linguistics; Computer Vision Systems; Consumption; Control Groups; Decision Making; Development; Diagnosis; Diagnostic; Dimensions; Disease; Face; Facial Expression; Frequencies; Funding; Furuncles; Future; Genetic study; Human; Individual; Individual Differences; Language; Left; Machine Learning; Measurement; Measures; Mental Health; Mental disorders; Methods; Modality; Modeling; Monitor; Motor; National Institute of Mental Health; National Institute on Deafness and Other Communication Disorders; Outcome Measure; Output; Participant; Persons; Phenotype; Pilot Projects; Public Health; Research; Sampling; Signal Transduction; Social Behavior; Social Interaction; Specificity; Stream; Techniques; Teenagers; Testing; Time; autism spectrum disorder; automated speech recognition; clinical care; clinical outcome measures; cognitive process; data fusion; design; digital; dyadic interaction; effective therapy; genetic association; improved; individuals with autism spectrum disorder; machine learning classification; machine learning classifier; machine learning model; meetings; motor behavior; multimodal data; multimodality; natural language; non-verbal; novel; precision medicine; skills; social; social communication; social deficits; sound; tool; trait; verbal

Abstract This proposal tackles an urgent need for sensitive clinical outcome measures of autism spectrum disorder (ASD) by developing an objective, digital, multi-modal social communication metric using computational linguistics (e.g., acoustic features, turn-taking rates, word frequency metrics). Our automatic speech recognition and natural language analytics approach is designed to fix known weaknesses in traditional measurements by providing granular information in less time, with built-in scalability for characterizing very large samples. Since ASD is defined by observables, it is ripe for an automated approach to digitizing behavior (e.g., words, sounds, facial expressions, motor behaviors). This proposal piggybacks on a recently funded R01 that uses computer vision and machine learning to characterize nonverbal motor synchrony in teens with either ASD or another disorder in a brief social conversation (MH118327, PI: Schultz). Vocal components of the conversation are not studied in MH118327; thus, the richness of the verbal domain is left untapped. We hypothesize that automatically derived spoken language markers will significantly predict group and individual differences in social communication skill, and – when fused with nonverbal features – will lead to better prediction than either modality alone. Together, these two projects represent a rare chance to study all observable social signals emitted during social interaction in the same diverse sample of participants. If funded, this project will be the first to use short conversations and multi-modal data fusion to predict social communication skill and diagnostic group in a large, clinically diverse sample of individuals with ASD and other disorders. Our pilot studies showed that a relatively small set of vocal features from a six-minute interaction predicts diagnosis (ASD vs. typical development [TD]) with 84% accuracy. These machine learning analyses also predicted social communication skill dimensionally, providing a granular metric of individual differences. Combining this approach with nonverbal metrics (R01MH118327) using decision level data fusion resulted in significantly better ASD vs. TD prediction – 91% accuracy. These pilot results are promising, but several gaps remain. In Aim 1 of this proposal, we assess the specificity of our vocal social communication approach by including a non-ASD psychiatric control group in our machine learning classification models, in addition to ASD and TD groups (N=250/group). In Aim 2, we clinically validate our transdiagnostic dimensional metric in a large, diverse sample of participants. In Aim 3, we test whether novel, sophisticated multi-modal fusion methods that combine vocal and nonverbal social communication features result in improved individual and group prediction. This proposal lays critical groundwork for an automated, precision medicine approach to studying, diagnosing, and caring for individuals with ASD and other mental health conditions. Suc- cessful completion of this project will transform how we quantify human behavior for a broad array of applications that demand efficient, scalable, and reliable measurement (e.g., genetic association studies, clinical trials and standard clinical care), thus meeting multiple strategic priorities set by NIMH and NIDCD.

摘要 这项建议解决了自闭症谱系障碍（ASD）敏感的临床结果措施的迫切需要 通过使用计算语言学开发一个客观的、数字化的、多模态的社会交流指标， (e.g.,声学特征、话轮转换率、词频度量）。我们的自动语音识别和自然 语言分析方法旨在通过提供 在更短的时间内获得粒度信息，具有内置的可扩展性，可用于表征非常大的样本。由于ASD是 由可观察量定义，对于数字化行为的自动化方法来说已经成熟（例如，语言、声音、面部表情 表情、运动行为）。该提案依赖于最近资助的使用计算机视觉的R 01 和机器学习来表征患有ASD或其他疾病的青少年的非语言运动同步 在一个简短的社会对话（MH 118327，PI：舒尔茨）。对话中的声音成分没有被研究， MH 118327;因此，语言领域的丰富性尚未开发。我们假设自动衍生的 口语标记语能显著预测社会交际技能的群体和个体差异， 当与非语言特征相结合时，将比单独使用任何一种方式更好地预测。在一起， 这两个项目代表了一个难得的机会，研究在社会交往中发出的所有可观察到的社会信号。 在同一个不同的参与者样本中。如果得到资助，这个项目将是第一个使用简短对话和 多模态数据融合，以预测大型临床多样化人群的社交技能和诊断组 患有ASD和其他疾病的个体的样本。我们的初步研究表明，一个相对较小的声音集， 来自六分钟互动的特征预测诊断（ASD与典型发展[TD]），准确率为84%。 这些机器学习分析还从维度上预测了社交沟通技能， 个体差异的度量。使用决策将此方法与非语言指标（R 01 MH 118327）结合 水平数据融合导致ASD与TD预测的显著更好- 91%的准确性。这些试点成果是 前景看好，但仍存在一些差距。在本提案的目标1中，我们评估了我们的语音社交的特异性， 通过在我们的机器学习分类中包括非ASD精神病对照组， 除ASD组和TD组外，模型组（N=250/组）。在目标2中，我们在临床上验证了我们的跨诊断 在一个大的，不同的参与者样本的维度度量。在目标3中，我们测试是否新颖，复杂 联合收割机结合声音和非语言社会交流特征的多模态融合方法导致改善的 个人和群体预测。该提案为自动化、精确医疗奠定了关键基础 研究，诊断和照顾患有ASD和其他心理健康状况的个人的方法。Suc- 这个项目的顺利完成将改变我们如何量化人类行为，以广泛的 需要高效、可扩展和可靠测量的应用（例如，遗传关联研究，临床 试验和标准临床护理），从而满足NIMH和NIDCD设定的多个战略优先事项。

项目成果

Conversational adaptation in children and teens with autism: Differences in talkativeness across contexts.

• DOI: 10.1002/aur.2693
• 发表时间: 2022-06
• 期刊: AUTISM RESEARCH
• 影响因子: 4.7
• 作者: Cola, Meredith;Zampella, Casey J.;Yankowitz, Lisa D.;Plate, Samantha;Petrulla, Victoria;Tena, Kimberly;Russell, Alison;Pandey, Juhi;Schultz, Robert T.;Parish-Morris, Julia
• 通讯作者: Parish-Morris, Julia

Natural language markers of social phenotype in girls with autism.

• DOI: 10.1111/jcpp.13348
• 发表时间: 2021-08
• 期刊: JOURNAL OF CHILD PSYCHOLOGY AND PSYCHIATRY
• 影响因子: 7.6
• 作者: Song, Amber;Cola, Meredith;Plate, Samantha;Petrulla, Victoria;Yankowitz, Lisa;Pandey, Juhi;Schultz, Robert T.;Parish-Morris, Julia
• 通讯作者: Parish-Morris, Julia

Personal victimization experiences of autistic and non-autistic children.

自闭症儿童和非自闭症儿童的个人受害经历。

• DOI: 10.1186/s13229-022-00531-4
• 发表时间: 2022-12-24
• 期刊: Molecular autism
• 影响因子: 6.2

Sex differences in friendships and loneliness in autistic and non-autistic children across development.

• DOI: 10.1186/s13229-023-00542-9
• 发表时间: 2023-02-24
• 期刊: Molecular autism
• 影响因子: 6.2

Friend matters: sex differences in social language during autism diagnostic interviews.

• DOI: 10.1186/s13229-021-00483-1
• 发表时间: 2022-01-10
• 期刊: Molecular autism
• 影响因子: 6.2
• 作者: Cola M;Yankowitz LD;Tena K;Russell A;Bateman L;Knox A;Plate S;Cubit LS;Zampella CJ;Pandey J;Schultz RT;Parish-Morris J
• 通讯作者: Parish-Morris J