SCH: INT: Collaborative Research: Dyadic Behavior Informatics for Psychotherapy Process and Outcome

SCH：INT：合作研究：心理治疗过程和结果的二元行为信息学

• 批准号: 1721667
• 负责人: Jeffrey Cohn
• 金额: $ 56.8万
• 依托单位: University of Pittsburgh
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1721667&HistoricalAwards=false
• 关键词: SCH; INT; Collaborative; Research; Dyadic

Using multimodal indicators, this project will develop a novel computational framework that models individual and interpersonal behavior in relation to process and outcomes in psychotherapy and other interpersonal contexts. The unique aspect of the project is the explicit joint and dyadic modeling of individuals' multimodal behaviors to holistically understand the system of the dyad. This research will pave the way to a better understanding of the dyadic behavior dynamics in psychotherapy and beyond. The project will build the computational foundations to predict process and outcomes, and more broadly inform behavioral science: The project will (1) contribute to knowledge about the psychotherapeutic process by identifying and characterizing behavior indicators with respect to process and outcome measures; (2) deepen our understanding of dyadic coordination dynamics that contribute to strong working alliance between clients and therapists; (3) make available to the research and clinical communities the Dyadic Behavior Informatics framework and Behavior Indicator Knowledgebase for use in other settings; and (4) establish the foundation for novel education and training materials and interventions. The knowledge and computational tools developed as part of this project will impact computing and behavioral science and applied domains more broadly. This project will advance understanding of dyadic behavioral dynamics by developing computational representations that can model fine-grained dyadic coordination between individuals and new algorithms that can model multi-level dynamics. Central to this research effort is the creation of the Behavior Indicator Knowledgebase (BIK) that will summarize discovered knowledge about significant and validated dyadic behavior indicators. While this work could have profound impact on behavioral and social science as a whole, the project specifically focuses on understanding the dynamics that predict process and outcome variables in psychotherapy. The project identifies five fundamental research challenges and presents a plan to address them directly: (1) Acquire a large, dyadic, and multimodal dataset of 64 patients with distress disorders seen over 8 psychotherapy sessions; (2) Create multimodal behavior indicators which can model the within session dynamics of the client or therapist; (3) Develop new dyadic behavior indicators that explicitly model the client-therapist coordination, (4) Develop abstract dyadic behavior representations that can learn the fine-grained dynamics between client and therapist behaviors; and, (5) Validate the computational representations (embeddings) and prediction models by assessing their impact on the predictive power of process and outcome measures in psychotherapy and assess generalizability beyond psychotherapy.

利用多模态指标，该项目将开发一种新的计算框架，对心理治疗和其他人际环境中与过程和结果相关的个人和人际行为进行建模。该项目的独特之处在于对个体的多模态行为进行明确的联合和二元建模，以全面了解二元系统。这项研究将为更好地理解心理治疗及其他领域的二元行为动力学铺平道路。该项目将建立预测过程和结果的计算基础，并更广泛地为行为科学提供信息：该项目将(1)通过识别和描述与过程和结果测量相关的行为指标，有助于了解心理治疗过程；(2)加深我们对有助于来访者和治疗师之间建立强大工作联盟的二元协调动力学的理解；(3)向研究和临床界提供二元行为信息学框架和行为指标知识库，以供其他环境使用；(4)为新型教育培训材料和干预措施奠定基础。作为该项目一部分的知识和计算工具将更广泛地影响计算和行为科学以及应用领域。该项目将通过开发计算表示来促进对二元行为动力学的理解，该计算表示可以模拟个体之间的细粒度二元协调，以及可以模拟多层次动力学的新算法。本研究工作的核心是创建行为指标知识库（BIK），该知识库将总结有关重要和已验证的二元行为指标的发现知识。虽然这项工作可能对整个行为和社会科学产生深远的影响，但该项目特别侧重于了解心理治疗中预测过程和结果变量的动态。该项目确定了五个基础研究挑战，并提出了一个直接解决这些挑战的计划：(1)获取一个大型的、双元的、多模态的数据集，其中包括64名经历过8次心理治疗的痛苦障碍患者；(2)创建多模态行为指标，可以模拟来访者或治疗师的会话动态；(3)开发新的二元行为指标，明确地模拟来访者和治疗师之间的协调；(4)开发抽象的二元行为表征，可以学习来访者和治疗师行为之间的细粒度动态；(5)通过评估其对心理治疗过程和结果测量的预测能力的影响，验证计算表征（嵌入）和预测模型，并评估其在心理治疗之外的推广能力。

项目成果

Automated Measurement of Head Movement Synchrony during Dyadic Depression Severity Interviews.

在二元抑郁严重程度访谈期间自动测量头部运动同步性。

• DOI: 10.1109/fg.2019.8756509
• 发表时间: 2019
• 期刊: Proceedings of the ... International Conference on Automatic Face and Gesture Recognition. IEEE International Conference on Automatic Face & Gesture Recognition
• 作者: Bhatia,Shalini;Goecke,Roland;Hammal,Zakia;Cohn,JeffreyF
• 通讯作者: Cohn,JeffreyF

FACS3D-Net: 3D Convolution based Spatiotemporal Representation for Action Unit Detection

• DOI: 10.1109/acii.2019.8925514
• 发表时间: 2019-09
• 期刊: 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)
• 作者: Le Yang;Itir Onal Ertugrul;J. Cohn;Z. Hammal;D. Jiang;H. Sahli