Automated assessment of dyadic interaction using physiological synchrony and machine learning

点击翻译按钮获取中文摘要

ABSTRACT Interpersonal communication is critical for human health and wellbeing in situations such as mental health intervention, education, and conflict resolution. However, assessment of communication quality and social connectedness continues to rely on self-report measures and subjective observations. A more objective and dynamic approach to the evaluation of interpersonal engagement could provide a useful complement to state- of-the-art methods. For example, alternative methods could allow researchers to better quantify the flow of social interaction, determine how different aspects of communication influence outcomes, and identify avenues for enhancing interpersonal communication. Furthermore, valid measures of social engagement could benefit related fields such as computer-supported collaboration, with potential broad impacts on human quality of life. Recent technological advances have enabled the study of physiological synchrony: a phenomenon in which the physiological responses of two individuals (e.g., heart rate, respiration) converge as the individuals interact. Synchronization occurs involuntarily and could provide rich information about the dynamics of interpersonal relationships. However, while studies have shown robust correlations between physiological synchrony and engagement at the group level, there has been practically no effort to use synchrony to assess engagement at the level of individual dyads. Thus, this project will develop and evaluate machine learning technologies that can automatically recognize mental/interpersonal states of individual dyads based on their physiological responses. The project will consist of two studies. In the first study, we will use regression algorithms to estimate dyadic engagement over 60-second intervals of a naturalistic 15-minute conversation. In the second study, we will then use classification algorithms to classify 4-minute acted conversation scenarios into one of 4 classes: positive two-sided, negative two-sided, and two one-sided conversation classes. Regression and classification represent two major families of machine learning techniques, each with advantages and disadvantages, and will thus be examined in complementary studies. For each study, five physiological measurements (electrocardiography, skin conductance, respiration, skin temperature, dry electroencephalography) will be collected from both members of the dyad to serve as the basis for regression and classification. Upon completion, the project will provide the research community with validated methods for extracting dyad- level information about interpersonal interaction from physiological measurements. This will pave the way for future research that could explore how physiology-based assessment could provide useful data in realistic scenarios (e.g., mental health intervention and education), how it could be combined with other techniques (e.g., self-report), and how it might be used to enhance interpersonal interaction. In the long term, automated analysis of physiological responses may become part of an efficient toolbox for analysis and enhancement of dyadic interaction, providing numerous benefits to human health, abilities, and well-being.