Predicting Autism and Social Functioning from Computer Vision Analyses of Motor Synchrony During Dyadic Interactions

通过计算机视觉对二元交互过程中运动同步的分析来预测自闭症和社交功能

• 批准号: 10057391
• 负责人: ROBERT Thomas SCHULTZ
• 金额: $ 72.09万
• 依托单位: CHILDREN'S HOSP OF PHILADELPHIA
• 依托单位国家: 美国
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-01-01 至 2023-11-30
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10057391
• 关键词: Address; Adolescent; Adult; Agreement; Base of the Brain; Behavior; Behavioral; Biologic Development; Biology; Categories; Child; Classification; Clinical; Clinical Trials; Code; Computer Vision Systems; Consumption; Data; Data Set; Diagnosis; Diagnostic; Dimensions; Disease; Face; Foundations; Genomics; Genotype; Goals; Grain; Grant; Human; Individual Differences; Limb structure; Machine Learning; Measurement; Measures; Mental disorders; Methodology; Methods; Modeling; Motion; Motor; Movement; Performance; Persons; Phenotype; Play; Property; Public Health; Questionnaires; Reporting; Reproducibility; Research; Role; Sampling; Scheme; Schools; Severities; Social Behavior; Social Functioning; Social Interaction; Specificity; Statistical Models; Symptoms; Techniques; Technology; Testing; Time; Training; Video Recording; Work; autism spectrum disorder; autistic children; behavioral phenotyping; computer framework; computerized; cost; design; digital; dyadic interaction; experimental study; impression; individual variation; informant; innovation; insight; lens; machine learning method; natural language; neuroimaging; novel; novel strategies; precision medicine; predictive modeling; skills; social; social communication; social deficits; tool; wearable sensor technology; young adult

PROJECT SUMMARY Our innovative and fully automated approach to the analysis of social behavior addresses the pressing need for precise and scalable measurements of the autism spectrum disorder (ASD) phenotype. Using computer vision and machine learning methods, we have created a novel, quantitative method for fine-grained analysis of social interactions. Our approach directly measures interpersonal motor synchrony, a construct which we use as a lens for understanding the social interaction differences that are at the core of ASD. Significance: Genomics and neuroimaging methods continuously evolve, providing deeper insights into the biology of ASD. However, methods for measuring the outward manifestations of ASD have not changed substantially in decades. ASD is fundamentally a disorder of social interaction, but current clinical tools do not directly measure observable social interactions. Instead, they summarize global impressions of these interactions via informant report questionnaires or observational coding schemes that typically lack the behavioral granularity needed to robustly measure individual differences and changes across time (e.g., treatment related change). Inter-rater agreement on questionnaires is typically modest, while the alternative “deep phenotyping” by expert clinicians is a time- consuming and often cost-prohibitive burden to studies, especially when large samples are required (e.g., in genomics research). Approach: To resolve these problems, our team created a novel computational framework that leverages advances in markerless video motion capture, computer vision, and machine learning to directly capture dyadic social interactions. This allows us to capture all behaviors observable by expert clinicians but with exquisite digital precision and objectivity. Preliminary Data: We developed a fully automatic quantitative assessment of interpersonal social behavior focused on features of dyadic facial motor synchrony. When applied to videos of brief conversations between confederates and young adults with or without ASD, our assessment predicted diagnostic status with 91% accuracy – significantly better than highly trained clinical experts assessing the same video recordings. The set of predictive social motor synchrony features that we identified also correlated significantly with symptom severity in the ASD group, suggesting that it can be used for both diagnostic classification and evaluating individual differences (vital for advancing precision medicine goals). Importantly, our findings were reproducible across samples: the same features identified in our adult analysis also predicted diagnosis in a child sample with high accuracy. Aims. In Aim 1, we will test the specificity of our computer vision approach by expanding comparisons to include a mixed psychiatric disorder group; Aim 2 will test dyadic synchrony in other body movements, and Aim 3 will define associations between interpersonal motor synchrony and dimensional aspects of social communication that span diagnostic categories. Impact: Our approach is designed for fast and rigorous assessment of social communication, providing a scalable solution to diagnosing ASD diagnosis and measuring individual variability, within a transdiagnostic, precision medicine framework.

项目摘要 我们对社会行为分析的创新和完全自动化的方法解决了紧迫的 需要对自闭症谱系障碍（ASD）表型的精确测量。使用 计算机视觉和机器学习方法，我们创建了一种新颖的定量方法，用于细粒度 社会互动的分析。我们的方法直接测量人际电机同步，这是一种结构 我们用作镜头来理解ASD核心的社会互动差异。意义： 基因组学和神经影像学方法不断发展，为ASD的生物学提供了更深入的见解。 但是，在数十年中，测量ASD的外部表现的方法并没有发生很大变化。 从根本上说，ASD是一种社会互动的障碍，但是当前的临床工具不能直接衡量可观察到的 社交互动。相反，他们通过线人报告总结了这些互动的全球印象 问卷或观察编码方案通常缺乏鲁棒性所需的行为粒度 测量跨时间的个体差异和变化（例如，与治疗相关的变化）。评估者协议 在问卷上通常是适度的，而专家临床医生的替代性“深层表型”是一个时代 - 对研究的消耗且经常过于过于过于过度的烧伤，尤其是在需要大型样本的情况下（例如，在 基因组学研究）。方法：解决这些问题，我们的团队创建了一个新颖的计算框架 这利用无标记的视频动作捕获，计算机视觉和机器学习的进步直接 捕获二元社会互动。这使我们能够捕获专家临床医生可观察的所有行为，但 精致的数字精度和目标。初步数据：我们开发了一个全自动定量 评估人际社会行为的重点是二元面部运动同步的特征。应用时 到有或没有ASD的同盟和年轻人之间的简短对话视频，我们的评估 预测的诊断状态为91％的准确性 - 明显好于训练有素的临床专家评估 相同的视频录音。我们也确定的一组预测社交电动机同步功能 与ASD组的症状严重程度显着相关，表明它可用于两种诊断 分类和评估个体差异（对于推进精确医学目标至关重要）。重要的是， 我们的发现可在样本中重现：我们的成人分析中确定的相同特征也预测了 在儿童样本中的诊断精度很高。目标。在AIM 1中，我们将测试计算机视觉的特异性 通过扩展比较以包括混合精神疾病组来进行方法； AIM 2将测试二元 其他身体运动中的同步，AIM 3将定义人际电机同步之间的关联 跨越诊断类别的社会沟通的维度方面。影响：我们的方法是 专为对社会交流的快速和严格评估而设计，为诊断提供了可扩展的解决方案 在经诊断，精确的医学框架内，ASD诊断和测量个体变异性。