Automated assessment of dyadic interaction using physiological synchrony and machine learning

摘要 人际沟通对人类健康和福祉至关重要，例如心理健康 干预、教育和解决冲突。然而，评估通信质量和社会 连通性仍然依赖于自我报告措施和主观观察。更客观和 对人际交往的动态评估方法可以为国家- 最先进的方法例如，替代方法可以使研究人员更好地量化社会流动， 互动，确定沟通的不同方面如何影响结果，并确定 加强人际沟通。此外，有效的社会参与措施可以受益于 相关领域，如计算机支持的合作，对人类生活质量的潜在广泛影响。 最近的技术进步使得对生理同步的研究成为可能： 两个个体的生理反应（例如，心率、呼吸）随着个体的交互而收敛。 同步是不自觉地发生的，可以提供有关人际关系动态的丰富信息。 关系。然而，尽管研究表明生理同步性和 在团队层面上的参与，实际上没有努力使用同步来评估参与， 个体的水平。因此，该项目将开发和评估机器学习技术， 自动识别基于他们的生理反应的个体二分体的精神/人际状态。 该项目将包括两项研究。在第一项研究中，我们将使用回归算法来估计并矢 在15分钟的自然对话中，每隔60秒进行一次互动。在第二项研究中，我们将 使用分类算法将4分钟的表演会话场景分为以下4类： 双面、负双面和两个单面会话类。回归和分类表示 机器学习技术的两大家族，每一个都有优点和缺点，因此， 在补充研究中检查。对于每项研究，五个生理测量（心电图， 皮肤电传导、呼吸、皮肤温度、干脑电图）将从两个 二分体的成员作为回归和分类的基础。 完成后，该项目将为研究界提供有效的方法来提取二分体， 从生理测量中获得有关人际互动的水平信息。这将为 未来的研究，可以探索如何基于生理学的评估可以提供有用的数据，在现实中， 场景（例如，心理健康干预和教育），如何与其他技术相结合（例如， 自我报告），以及如何使用它来加强人际互动。从长远来看，自动分析 生理反应的分析和增强可能成为一个有效的工具箱的一部分 互动，为人类健康，能力和福祉提供许多好处。

ABSTRACT Interpersonal communication is critical for human health and wellbeing in situations such as mental health intervention, education, and conflict resolution. However, assessment of communication quality and social connectedness continues to rely on self-report measures and subjective observations. A more objective and dynamic approach to the evaluation of interpersonal engagement could provide a useful complement to state- of-the-art methods. For example, alternative methods could allow researchers to better quantify the flow of social interaction, determine how different aspects of communication influence outcomes, and identify avenues for enhancing interpersonal communication. Furthermore, valid measures of social engagement could benefit related fields such as computer-supported collaboration, with potential broad impacts on human quality of life. Recent technological advances have enabled the study of physiological synchrony: a phenomenon in which the physiological responses of two individuals (e.g., heart rate, respiration) converge as the individuals interact. Synchronization occurs involuntarily and could provide rich information about the dynamics of interpersonal relationships. However, while studies have shown robust correlations between physiological synchrony and engagement at the group level, there has been practically no effort to use synchrony to assess engagement at the level of individual dyads. Thus, this project will develop and evaluate machine learning technologies that can automatically recognize mental/interpersonal states of individual dyads based on their physiological responses. The project will consist of two studies. In the first study, we will use regression algorithms to estimate dyadic engagement over 60-second intervals of a naturalistic 15-minute conversation. In the second study, we will then use classification algorithms to classify 4-minute acted conversation scenarios into one of 4 classes: positive two-sided, negative two-sided, and two one-sided conversation classes. Regression and classification represent two major families of machine learning techniques, each with advantages and disadvantages, and will thus be examined in complementary studies. For each study, five physiological measurements (electrocardiography, skin conductance, respiration, skin temperature, dry electroencephalography) will be collected from both members of the dyad to serve as the basis for regression and classification. Upon completion, the project will provide the research community with validated methods for extracting dyad- level information about interpersonal interaction from physiological measurements. This will pave the way for future research that could explore how physiology-based assessment could provide useful data in realistic scenarios (e.g., mental health intervention and education), how it could be combined with other techniques (e.g., self-report), and how it might be used to enhance interpersonal interaction. In the long term, automated analysis of physiological responses may become part of an efficient toolbox for analysis and enhancement of dyadic interaction, providing numerous benefits to human health, abilities, and well-being.