Collaborative Research: SCH: Multimodal Algorithms for Motor Imitation Assessment in Children with Autism

在美国，大约每 54 名儿童中就有 1 名被诊断患有自闭症谱系障碍 (ASD)。鉴于其高患病率，需要一种自动且可扩展的方法来为诊断和行为治疗提供信息。虽然之前寻找早期出现的、可靠的自闭症谱系障碍定量生物标志物的工作主要集中在非运动特征上，但大量的研究证据揭示了许多自闭症谱系障碍儿童的运动模仿受损模式，这使得运动模仿缺陷成为寻找表型生物标志物的一个有希望的途径。然而，传统的仿品评估方法往往依赖专家观察，成本高、耗时长、容易出错，且缺乏客观性和可扩展性。计算机视觉和机器学习的最新进展使人工智能成为一种有前景的技术，可以设计客观、可重复且高度可扩展的多模式系统，该系统不仅可以在设备齐全的临床环境中运行，而且可以在家中评估自闭症谱系障碍儿童的模仿能力。然而，需要解决一些关键挑战，例如用于 ASD 评估的特定模仿任务的设计、用于训练机器学习算法的多模态数据的收集和标记、以及开发用于比较此类运动的新颖的细粒度表示人类运动和指标，以测试自动运动模仿评估算法的有效性、可扩展性和可重复性，以告知 ASD 诊断。该项目的总体目标是设计、开发和测试一个客观、可重复和高度可扩展的多模态系统，以观察儿童执行简短视频类似游戏的运动模仿任务，定量评估他们的运动模仿表现，并研究其作为自闭症表型生物标志物的有效性。实现这一目标需要采用跨学科方法，结合自闭症、儿童发展、计算机视觉和机器学习方面的专业知识。具体来说，该项目将：（1）设计与 ASD 评估相关的运动模仿任务，（2）设计、测试和验证一个可扩展且灵活的系统，以收集和标记儿童模仿一系列动作的多模态数据； （3）设计一种新颖的细粒度人体动作表示，可以有效学习，并且适合将儿童的动作与他们需要模仿的动作进行比较； (4) 开发新颖的计算机视觉和度量学习算法，用于学习和比较人类运动的多模态表示，以及 (5) 使用此类度量来生成候选模仿分数，该分数可用作 ASD 的潜在定量生物标志物。该项目中开发的运动模仿评估方法除了评估自闭症谱系障碍儿童外，还可用于多种应用，例如为基于视频的康复治疗、手术技能评估、体育活动和其他基于运动的教学活动提供模仿表现分数。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Approximately 1 in 54 children in the US is diagnosed with autism spectrum disorder (ASD). Given its high prevalence, there is a need for an automatic and scalable method to inform diagnosis and behavioral therapies. While prior work on finding early-emerging and reliable quantitative biomarkers of ASD has focused on non-motor features, abundant research evidence has revealed patterns of impaired motor imitation in a wide range of children with ASD, making motor imitation deficits a promising avenue to find a phenotypic biomarker. However, traditional imitation assessment methods often rely on expert-based observation, which is costly, time-consuming and error-prone, and lacks objectivity and scalability. Recent advances in computer vision and machine learning make artificial intelligence a promising technology to design an objective, reproducible and highly-scalable multimodal system functioning not only in well-equipped clinical setups but also at home for assessing imitation performance in children with ASD. However, critical challenges such as the design of specific imitation tasks for ASD assessment, the collection and labeling of multimodal data for training machine learning algorithms, and the development of novel fine-grained representations human movements and metrics for comparing such movements need to be addressed to test the validity, scalability and reproducibility of automatic motor imitation assessment algorithms to inform ASD diagnosis.The overall goal of this project is to design, develop and test an objective, reproducible and highly-scalable multimodal system to observe children performing a brief video game-like motor imitation task, quantitatively assess their motor imitation performance, and investigate its validity as a phenotypic biomarker for autism. Accomplishing this goal will require an interdisciplinary approach which combines expertise in autism, child development, computer vision and machine learning. Specifically, this project will: (1) design motor imitation tasks that are relevant for ASD assessment, (2) design, test and validate a scalable and flexible system to collect and label multimodal data of children imitating a sequence of movements; (3) design a novel fine-grained representation of human movements that can be learned efficiently and is suitable for comparing the children's movements to the movements they need to imitate; (4) develop novel computer vision and metric learning algorithms for learning and comparing multimodal representations of human movements, and (5) use such metrics to generate candidate imitations scores that can be used as potential quantitative biomarkers for ASD. The motor imitation assessment methods to be developed in this project could be used in a wide variety of applications beyond assessing children with ASD, such as providing imitation performance scores for video-based rehabilitation therapy, surgical skill assessment, athletic activities and other movement-based instructional activities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.