Collaborative Research: SCH: Multimodal Algorithms for Motor Imitation Assessment in Children with Autism

在美国，大约每54名儿童中就有一名被诊断出患有自闭症谱系障碍(ASD)。鉴于其高患病率，需要一种自动且可扩展的方法来为诊断和行为疗法提供信息。虽然之前寻找ASD早期出现和可靠的定量生物标志物的工作主要集中在非运动特征上，但大量的研究证据表明，在广泛的ASD儿童中，运动模仿受损的模式，使运动模仿缺陷成为寻找表型生物标志物的有希望的途径。然而，传统的模仿评估方法往往依赖于基于专家的观察，成本高、耗时长、容易出错，缺乏客观性和可扩展性。计算机视觉和机器学习的最新进展使人工智能成为一种很有前途的技术，可以设计一个客观的、可重复的和高度可扩展的多模式系统，不仅可以在设备齐全的临床设备中运行，而且还可以在家里评估自闭症儿童的模仿表现。然而，需要解决的关键挑战包括为ASD评估设计特定的模仿任务，为训练机器学习算法收集和标记多模式数据，开发新的细粒度表示人体动作和比较此类动作的度量标准，以测试自动运动模仿评估算法的有效性、可扩展性和重复性，从而为ASD诊断提供信息。本项目的总体目标是设计、开发和测试一个客观、可重复性和高度可伸缩性的多模式系统，以观察儿童执行简短的视频游戏般的运动模仿任务，定量评估他们的运动模仿表现，并调查其作为自闭症表型生物标志物的有效性。实现这一目标将需要一种跨学科的方法，将自闭症、儿童发展、计算机视觉和机器学习方面的专业知识结合起来。具体地说，这个项目将：(1)设计与ASD评估相关的运动模仿任务；(2)设计、测试和验证一个可扩展和灵活的系统来收集和标记儿童模仿一系列动作的多模式数据；(3)设计一个新的细粒度的人体动作表示，该表示可以有效地学习，并适合于将儿童的动作与他们需要模仿的动作进行比较；(4)开发新的计算机视觉和度量学习算法，用于学习和比较人类动作的多模式表示，以及(5)使用这些度量来生成候选模仿分数，可用作ASD的潜在定量生物标记。在这个项目中开发的运动模仿评估方法除了评估患有自闭症的儿童外，还可以用于广泛的应用，例如为基于视频的康复治疗、手术技能评估、运动活动和其他基于运动的教学活动提供模仿表现分数。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Approximately 1 in 54 children in the US is diagnosed with autism spectrum disorder (ASD). Given its high prevalence, there is a need for an automatic and scalable method to inform diagnosis and behavioral therapies. While prior work on finding early-emerging and reliable quantitative biomarkers of ASD has focused on non-motor features, abundant research evidence has revealed patterns of impaired motor imitation in a wide range of children with ASD, making motor imitation deficits a promising avenue to find a phenotypic biomarker. However, traditional imitation assessment methods often rely on expert-based observation, which is costly, time-consuming and error-prone, and lacks objectivity and scalability. Recent advances in computer vision and machine learning make artificial intelligence a promising technology to design an objective, reproducible and highly-scalable multimodal system functioning not only in well-equipped clinical setups but also at home for assessing imitation performance in children with ASD. However, critical challenges such as the design of specific imitation tasks for ASD assessment, the collection and labeling of multimodal data for training machine learning algorithms, and the development of novel fine-grained representations human movements and metrics for comparing such movements need to be addressed to test the validity, scalability and reproducibility of automatic motor imitation assessment algorithms to inform ASD diagnosis.The overall goal of this project is to design, develop and test an objective, reproducible and highly-scalable multimodal system to observe children performing a brief video game-like motor imitation task, quantitatively assess their motor imitation performance, and investigate its validity as a phenotypic biomarker for autism. Accomplishing this goal will require an interdisciplinary approach which combines expertise in autism, child development, computer vision and machine learning. Specifically, this project will: (1) design motor imitation tasks that are relevant for ASD assessment, (2) design, test and validate a scalable and flexible system to collect and label multimodal data of children imitating a sequence of movements; (3) design a novel fine-grained representation of human movements that can be learned efficiently and is suitable for comparing the children's movements to the movements they need to imitate; (4) develop novel computer vision and metric learning algorithms for learning and comparing multimodal representations of human movements, and (5) use such metrics to generate candidate imitations scores that can be used as potential quantitative biomarkers for ASD. The motor imitation assessment methods to be developed in this project could be used in a wide variety of applications beyond assessing children with ASD, such as providing imitation performance scores for video-based rehabilitation therapy, surgical skill assessment, athletic activities and other movement-based instructional activities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.