Data-driven multidimensional modeling of nonverbal communication in typical and atypical development

点击翻译按钮获取中文摘要

ABSTRACT This proposal will develop innovative technology for data-driven, multimodal characterization of nonverbal communication (NVC) in typical and atypical development. Prior research has provided qualitative descriptions of the development of children's use of gaze and gesture to regulate social interactions, but there are no objective, automated tools for measuring NVC behaviors, nor computational models to explain their coordination and timing in social interactions. This proposal will apply advanced probabilistic modeling techniques from machine learning and data mining to a rich corpus of children's behavior, including automated measures of children's posture, head pose, gaze direction, arm movements, and hand configurations derived from color and depth cameras and accelerometers. By automatically learning probabilistic latent variable models from movement data, we will obtain compact, data-driven descriptions of NVC and its coordination in children with autism, children with developmental delays without autism, and typically developing children (Aim 1). We will validate our models by demonstrating their ability to predict children's behavior, including diagnostic group and one-year language outcomes (Aim 2). We will test whether novel NVC patterns can be uncovered with bottom-up clustering of motor movement data (Aim 3). We predict our models will have greater explanatory and predictive power compared to current measures of NVC, which are typically human-coded behaviors that are descriptive, but rely on a-priori definitions of higher level behaviors. The models we develop will capture the fine-grained structure, coordination, and timing of NVC behaviors during social interactions, and thus allow us to characterize these behaviors with an unprecedented level of detail. Because interventions for young children with ASD target NVC skills, our automated measurement tools will provide clinicians with powerful new tools to assess the extent to which these treatments are efficacious. In addition, automated tools for dense measurement of fine-grained changes in NVC would enable clinicians to assess profiles of strengths and weaknesses for purposes of treatment planning, to dynamically tailor treatment to clients' changing abilities, and ultimately to accurately capture whether treatment is working. Finally, the measurement capabilities will provide researchers with a novel, cost-effective approach to analyze video recordings, at a scale that is not currently feasible due to a reliance on human coding.