Automated assessment of dyadic interaction using physiological synchrony and machine learning

使用生理同步和机器学习自动评估二元相互作用

• 批准号: 10353119
• 负责人: Vesna Dominika Novak
• 金额: $ 7.34万
• 依托单位: UNIVERSITY OF CINCINNATI
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-02-01 至 2024-01-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10353119
• 关键词: Algorithms; Brain; Categories; Classification; Client; Clinical Psychology; Collaborations; Communication; Communities; Complement; Computers; Data; Data Set; Disadvantaged; Education; Electrocardiogram; Electroencephalography; Emotional; Evaluation; Family; Foundations; Galvanic Skin Response; Goals; Health; Heart Rate; Human; Individual; Interpersonal Relations; Intervention; Machine Learning; Measurement; Measures; Mental Health; Methods; Outcome; Participant; Patient Self-Report; Pattern Recognition; Performance; Personal Communication; Personal Satisfaction; Persons; Physiological; Physiology; Psyche structure; Public Health; Quality of life; Research; Research Personnel; Research Project Grants; Respiration; Side; Signal Transduction; Skin Temperature; Social Functioning; Social Interaction; Source; Techniques; Technology; Testing; Time; automated analysis; base; classification algorithm; conflict resolution; dyadic interaction; improved; innovative technologies; insight; machine learning algorithm; member; mental health counseling; multidisciplinary; regression algorithm; response; sensor; signal processing; social; social engagement; social relationships; statistics; student participation; teacher; time use; tool

ABSTRACT Interpersonal communication is critical for human health and wellbeing in situations such as mental health intervention, education, and conflict resolution. However, assessment of communication quality and social connectedness continues to rely on self-report measures and subjective observations. A more objective and dynamic approach to the evaluation of interpersonal engagement could provide a useful complement to state- of-the-art methods. For example, alternative methods could allow researchers to better quantify the flow of social interaction, determine how different aspects of communication influence outcomes, and identify avenues for enhancing interpersonal communication. Furthermore, valid measures of social engagement could benefit related fields such as computer-supported collaboration, with potential broad impacts on human quality of life. Recent technological advances have enabled the study of physiological synchrony: a phenomenon in which the physiological responses of two individuals (e.g., heart rate, respiration) converge as the individuals interact. Synchronization occurs involuntarily and could provide rich information about the dynamics of interpersonal relationships. However, while studies have shown robust correlations between physiological synchrony and engagement at the group level, there has been practically no effort to use synchrony to assess engagement at the level of individual dyads. Thus, this project will develop and evaluate machine learning technologies that can automatically recognize mental/interpersonal states of individual dyads based on their physiological responses. The project will consist of two studies. In the first study, we will use regression algorithms to estimate dyadic engagement over 60-second intervals of a naturalistic 15-minute conversation. In the second study, we will then use classification algorithms to classify 4-minute acted conversation scenarios into one of 4 classes: positive two-sided, negative two-sided, and two one-sided conversation classes. Regression and classification represent two major families of machine learning techniques, each with advantages and disadvantages, and will thus be examined in complementary studies. For each study, five physiological measurements (electrocardiography, skin conductance, respiration, skin temperature, dry electroencephalography) will be collected from both members of the dyad to serve as the basis for regression and classification. Upon completion, the project will provide the research community with validated methods for extracting dyad- level information about interpersonal interaction from physiological measurements. This will pave the way for future research that could explore how physiology-based assessment could provide useful data in realistic scenarios (e.g., mental health intervention and education), how it could be combined with other techniques (e.g., self-report), and how it might be used to enhance interpersonal interaction. In the long term, automated analysis of physiological responses may become part of an efficient toolbox for analysis and enhancement of dyadic interaction, providing numerous benefits to human health, abilities, and well-being.

抽象的 人际沟通对于心理健康等情况下的人类健康和福祉至关重要 干预、教育和解决冲突。然而，对沟通质量和社交的评估 连通性仍然依赖于自我报告措施和主观观察。更加客观和 评估人际参与的动态方法可以为国家评估提供有用的补充 最先进的方法。例如，替代方法可以让研究人员更好地量化社交流量 互动，确定沟通的不同方面如何影响结果，并确定沟通的途径 加强人际沟通。此外，有效的社会参与措施可以受益 相关领域，例如计算机支持的协作，对人类生活质量具有潜在的广泛影响。 最近的技术进步使得生理同步性的研究成为可能：一种现象，其中 当两个人互动时，他们的生理反应（例如心率、呼吸）会趋同。 同步是不由自主地发生的，可以提供有关人际动态的丰富信息 关系。然而，虽然研究表明生理同步性和 尽管在集团层面的参与度方面，实际上还没有尝试使用同步性来评估集团层面的参与度。 个体二元组的水平。因此，该项目将开发和评估机器学习技术，这些技术可以 根据个体的生理反应自动识别个体的心理/人际状态。 该项目将包括两项研究。在第一项研究中，我们将使用回归算法来估计二元 在 15 分钟的自然对话中，有超过 60 秒的参与时间。在第二项研究中，我们将 使用分类算法将 4 分钟的对话场景分为 4 类之一：正面 双边、消极双边和两个单边对话类。回归和分类代表 机器学习技术的两大家族，各有优缺点，因此将被 在补充研究中进行了检查。对于每项研究，五项生理测量（心电图、 皮肤电导、呼吸、皮肤温度、干脑电图）将从两者中收集 二元组的成员作为回归和分类的基础。 完成后，该项目将为研究界提供经过验证的方法来提取二元组 从生理测量中获取有关人际互动的水平信息。这将为 未来的研究可以探索基于生理学的评估如何在现实中提供有用的数据 情景（例如，心理健康干预和教育），如何与其他技术相结合（例如， 自我报告），以及如何使用它来增强人际互动。从长远来看，自动化分析 生理反应的研究可能成为分析和增强二元反应的有效工具箱的一部分 互动，为人类健康、能力和福祉带来诸多益处。