Validation of a Virtual Still Face Procedure and Deep Learning Algorithms to Assess Infant Emotion Regulation and Infant-Caregiver Interactions in the Wild

“这项研究是NIH帮助结束成瘾长期(Heal)倡议的一部分，该倡议旨在加快科学解决国家阿片类药物公共健康危机。NIH Hear倡议支持整个NIH的研究，以改善阿片类药物滥用和成瘾的治疗。“ 每时每刻的婴儿与父母的互动是婴儿学习调节情绪的中心背景。研究情绪调节展开的婴儿与父母的相互作用对于有情绪失调和/或关系障碍风险的婴儿尤其重要，包括出生前接触物质的婴儿。然而，目前评估婴儿情绪调节和婴儿与父母互动的最先进方法主要依赖于简短的实验室任务。这些程序给参与者带来了负担，尤其是家庭。 经历人口和心理社会风险，并限制研究结果的概括性和生态有效性。(A)机器学习方法方面的技术进步，包括从原始的未标记数据中挖掘复杂模式的深度学习方法，以及(B)可穿戴传感器有可能改变我们捕捉婴儿在现实世界环境中时刻情绪体验的能力，同时也减轻参与婴儿研究的家庭的负担。考虑到这些问题，我们将开发下一代方法来评估婴儿情绪调节和婴儿与父母的互动。在此过程中，我们将使用 LittleBeats是我们团队开发的一款婴儿多模式可穿戴设备，用于在家中收集较长时间(约每天8-10小时)的婴儿和父母发声(通过麦克风)、婴儿运动活动(通过运动传感器)和婴儿心脏迷走神经张力(通过心电图)的时间同步数据。我们提出了三个具体目标。首先，我们将验证黄金标准仍然面孔范例的虚拟访问方案，该方案通常在实验室环境中进行，用于评估婴儿在第一年的情绪调节。其次，我们将验证多模式深度学习算法，以实时检测婴儿的情绪状态 使用LittleBeats音频、心电和运动数据。第三，我们将验证深度学习算法来检测和标记婴儿和父母的发声类型(喋喋不休、小题大做、哭闹、大笑)和父母(婴儿直接发声、成人发声、唱歌、大笑)，这些发声类型创建了婴儿与父母发声互动的构建块，如话轮转换。通过将创新的可穿戴技术与尖端的深度学习算法相结合，我们的目标是促进对产前物质暴露导致不良后果的机制的理解。此外，产前接触物质是一种异质现象，涉及环境风险和 保护因素，从而使一刀切的方法无效。通过监测婴儿情绪调节的每时每刻的变化，结合在婴儿表现出痛苦迹象的时刻检测和分类婴儿与父母互动的深度学习算法，所提出的方法有可能改变我们对产前物质暴露导致不良结局的动态过程的理解，并准确定位促进最佳发育的保护因素。

“This study is part of the NIH’s Helping to End Addiction Long-term (HEAL) initiative to speed scientific solutions to the national opioid public health crisis. The NIH HEAL Initiative bolsters research across NIH to improve treatment for opioid misuse and addiction.” Moment-to-moment infant-parent interactions are a central context in which infants learn to regulate emotions. Investigating infant-parent interactions in which emotion regulation unfolds is particularly important for infants at risk for emotion dysregulation and/or relationship disturbance, including infants with prenatal substance exposure. Yet, current state-of-the art methods to assess infant emotion regulation and infant-parent interaction predominantly rely on brief laboratory tasks. These procedures pose burdens on participants, especially families experiencing demographic and psychosocial risk, and place limits on generalizability and ecological validity of findings. Technological advances in (a) machine learning methods, including deep learning approaches that mine for complex patterns in raw unlabeled data, and (b) wearable sensors have the potential to transform our ability to capture infants’ moment-to-moment emotional experiences in their real-world environments, while also lowering burden on families participating in infant research. With these issues in mind, we will develop next-generation methods to assess infant emotion regulation and infant-parent interaction. In doing so, we will use LittleBeats, an infant multimodal wearable device developed by our team, to collect time-synced data on infant and parent vocalizations (via microphone), infant motor activity (via motion sensor), and infant cardiac vagal tone (via electrocardiogram [ECG]) for extended periods of time (~8-10 hours per day) in the home. We propose three specific aims. First, we will validate a virtual visit protocol for the gold-standard Still Face Paradigm, which is typically conducted in a laboratory setting, for assessing emotion regulation among infants during the first year of life. Second, we will validate multimodal deep learning algorithms to detect infant emotional states in real time using LittleBeats audio, ECG and motion data. Third, we will validate deep learning algorithms to detect and label vocalization types of infants (babble, fuss, cry, laugh) and parents (infant-direct speech, adult-directed speech, sing, laugh), which create the build blocks of infant-parent vocal interactions, such as turn taking. By bringing together innovative wearable technology with cutting-edge deep learning algorithms, we aim to advance understanding of the mechanisms through which prenatal substance exposures contribute to adverse outcomes. Further, prenatal substance exposure is a heterogeneous phenomena that transacts with environmental risk and protective factors, thereby making a one-size-fits-all approach ineffective. By monitoring moment-to-moment changes in infants’ emotion regulation, combined with deep learning algorithms that detect and classify infant-parent interactions during moments when infant show signs of distress, the proposed methods have the potential to transform our understanding of the dynamic processes through which prenatal substance exposure leads to poor outcomes and pinpoint protective factors that promote optimal development.