Data-driven multidimensional modeling of nonverbal communication in typical and atypical development

摘要 该提案将开发创新技术，用于数据驱动的非语言多模态表征 典型和非典型发展中的传播（NVC）。先前的研究提供了定性描述 儿童使用凝视和手势来调节社会交往的发展，但没有 客观的，自动化的工具来衡量NVC的行为，也没有计算模型来解释他们的行为。 社会互动中的协调和时机。该提案将采用先进的概率建模 从机器学习和数据挖掘技术到丰富的儿童行为语料库，包括自动化 测量儿童的姿势，头部姿势，注视方向，手臂运动和手的配置 从彩色和深度相机和加速计。通过自动学习概率潜变量 从运动数据模型，我们将获得紧凑的，数据驱动的描述NVC及其协调， 自闭症儿童、无自闭症的发育迟缓儿童和典型发育儿童（Aim 1）。我们将通过展示他们预测儿童行为的能力来验证我们的模型，包括诊断 组和一年的语言成果（目标2）。我们将测试是否可以发现新的NVC模式 自下而上聚类运动数据（目标3）。我们预测我们的模型会有更大的 解释和预测能力相比，目前的措施，NVC，这是典型的人类编码 描述性的行为，但依赖于更高级别行为的先验定义。 我们开发的模型将捕获NVC行为的细粒度结构、协调和时序 在社会交往中，从而使我们能够以前所未有的水平来描述这些行为。 详细由于对ASD幼儿的干预针对NVC技能，我们的自动测量工具 将为临床医生提供强大的新工具，以评估这些治疗的有效程度。在 此外，用于密集测量NVC细粒度变化的自动化工具将使临床医生能够 评估治疗计划的优势和劣势，以动态调整 我们的目标是根据客户不断变化的能力来进行治疗，并最终准确地捕捉治疗是否有效。 最后，测量能力将为研究人员提供一种新颖的、具有成本效益的分析方法。 视频记录，由于依赖于人类编码，目前还不可行。

ABSTRACT This proposal will develop innovative technology for data-driven, multimodal characterization of nonverbal communication (NVC) in typical and atypical development. Prior research has provided qualitative descriptions of the development of children's use of gaze and gesture to regulate social interactions, but there are no objective, automated tools for measuring NVC behaviors, nor computational models to explain their coordination and timing in social interactions. This proposal will apply advanced probabilistic modeling techniques from machine learning and data mining to a rich corpus of children's behavior, including automated measures of children's posture, head pose, gaze direction, arm movements, and hand configurations derived from color and depth cameras and accelerometers. By automatically learning probabilistic latent variable models from movement data, we will obtain compact, data-driven descriptions of NVC and its coordination in children with autism, children with developmental delays without autism, and typically developing children (Aim 1). We will validate our models by demonstrating their ability to predict children's behavior, including diagnostic group and one-year language outcomes (Aim 2). We will test whether novel NVC patterns can be uncovered with bottom-up clustering of motor movement data (Aim 3). We predict our models will have greater explanatory and predictive power compared to current measures of NVC, which are typically human-coded behaviors that are descriptive, but rely on a-priori definitions of higher level behaviors. The models we develop will capture the fine-grained structure, coordination, and timing of NVC behaviors during social interactions, and thus allow us to characterize these behaviors with an unprecedented level of detail. Because interventions for young children with ASD target NVC skills, our automated measurement tools will provide clinicians with powerful new tools to assess the extent to which these treatments are efficacious. In addition, automated tools for dense measurement of fine-grained changes in NVC would enable clinicians to assess profiles of strengths and weaknesses for purposes of treatment planning, to dynamically tailor treatment to clients' changing abilities, and ultimately to accurately capture whether treatment is working. Finally, the measurement capabilities will provide researchers with a novel, cost-effective approach to analyze video recordings, at a scale that is not currently feasible due to a reliance on human coding.