Validation of a Virtual Still Face Procedure and Deep Learning Algorithms to Assess Infant Emotion Regulation and Infant-Caregiver Interactions in the Wild

验证虚拟静脸程序和深度学习算法，以评估野外婴儿情绪调节和婴儿与护理人员的互动

• 批准号: 10777825
• 负责人: MARK ALLAN HASEGAWA-JOHNSON
• 金额: $ 62.28万
• 依托单位: UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-30 至 2028-07-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10777825
• 关键词: Achievement; Adult; Architecture; Behavior; Behavioral; Brain; Cardiac; Caregivers; Categories; Child; Classification; Code; Complex; Crying; Data; Data Set; Development; Devices; Dimensions; Distress; Early Diagnosis; Early Intervention; Electrocardiogram; Emotional; Emotions; Engineering; Environment; Environmental Risk Factor; Evaluation; Exposure to; Face; Family; Fathers; Helping to End Addiction Long-term; Home; Home environment; Hour; Household; Human; Infant; Infant Behavior; Infant Care; Intervention; Label; Laboratories; Learning; Life; Link; Machine Learning; Manuals; Measures; Methods; Mind; Modality; Modeling; Monitor; Mothers; Motion; Motor Activity; Outcome; Parents; Participant; Pattern; Persons; Phase; Physiological; Physiology; Play; Pregnancy; Procedures; Process; Protocols documentation; Research; Research Priority; Risk; Sampling; Science; Speech; Stress; Technology; Testing; Time; Training; Transact; Validation; Video Recording; Visit; adverse outcome; analytical method; analytical tool; biobehavior; clinical practice; cost; data streams; deep learning; deep learning algorithm; deep learning model; emotion dysregulation; emotion regulation; emotional experience; experience; family burden; fetal substance exposure; gaze; innovation; machine learning method; member; microphone; motion sensor; multimodal data; multimodality; next generation; prenatal; prenatal exposure; protective factors; psychosocial; remote assessment; sensor; signal processing; substance use; supervised learning; time use; virtual; virtual assessment; virtual visit; vocalization; wearable device; wearable platform; wearable sensor technology

“This study is part of the NIH’s Helping to End Addiction Long-term (HEAL) initiative to speed scientific solutions to the national opioid public health crisis. The NIH HEAL Initiative bolsters research across NIH to improve treatment for opioid misuse and addiction.” Moment-to-moment infant-parent interactions are a central context in which infants learn to regulate emotions. Investigating infant-parent interactions in which emotion regulation unfolds is particularly important for infants at risk for emotion dysregulation and/or relationship disturbance, including infants with prenatal substance exposure. Yet, current state-of-the art methods to assess infant emotion regulation and infant-parent interaction predominantly rely on brief laboratory tasks. These procedures pose burdens on participants, especially families experiencing demographic and psychosocial risk, and place limits on generalizability and ecological validity of findings. Technological advances in (a) machine learning methods, including deep learning approaches that mine for complex patterns in raw unlabeled data, and (b) wearable sensors have the potential to transform our ability to capture infants’ moment-to-moment emotional experiences in their real-world environments, while also lowering burden on families participating in infant research. With these issues in mind, we will develop next-generation methods to assess infant emotion regulation and infant-parent interaction. In doing so, we will use LittleBeats, an infant multimodal wearable device developed by our team, to collect time-synced data on infant and parent vocalizations (via microphone), infant motor activity (via motion sensor), and infant cardiac vagal tone (via electrocardiogram [ECG]) for extended periods of time (~8-10 hours per day) in the home. We propose three specific aims. First, we will validate a virtual visit protocol for the gold-standard Still Face Paradigm, which is typically conducted in a laboratory setting, for assessing emotion regulation among infants during the first year of life. Second, we will validate multimodal deep learning algorithms to detect infant emotional states in real time using LittleBeats audio, ECG and motion data. Third, we will validate deep learning algorithms to detect and label vocalization types of infants (babble, fuss, cry, laugh) and parents (infant-direct speech, adult-directed speech, sing, laugh), which create the build blocks of infant-parent vocal interactions, such as turn taking. By bringing together innovative wearable technology with cutting-edge deep learning algorithms, we aim to advance understanding of the mechanisms through which prenatal substance exposures contribute to adverse outcomes. Further, prenatal substance exposure is a heterogeneous phenomena that transacts with environmental risk and protective factors, thereby making a one-size-fits-all approach ineffective. By monitoring moment-to-moment changes in infants’ emotion regulation, combined with deep learning algorithms that detect and classify infant-parent interactions during moments when infant show signs of distress, the proposed methods have the potential to transform our understanding of the dynamic processes through which prenatal substance exposure leads to poor outcomes and pinpoint protective factors that promote optimal development.

“这项研究是NIH帮助结束长期成瘾（HEAL）计划的一部分，旨在加快国家阿片类药物公共卫生危机的科学解决方案。NIH HEAL倡议支持整个NIH的研究，以改善阿片类药物滥用和成瘾的治疗。 每时每刻的婴儿与父母的互动是婴儿学习调节情绪的中心背景。调查婴儿与父母的互动，其中情绪调节展开是特别重要的婴儿在情绪失调和/或关系障碍的风险，包括婴儿与产前物质暴露。然而，目前评估婴儿情绪调节和婴儿-父母互动的最先进的方法主要依赖于简单的实验室任务。这些程序给参与者，特别是家庭带来负担 经历人口和心理社会风险，并限制调查结果的普遍性和生态有效性。（a）机器学习方法的技术进步，包括在原始未标记数据中挖掘复杂模式的深度学习方法，以及（B）可穿戴传感器有可能改变我们捕捉婴儿在现实世界环境中每时每刻情感体验的能力，同时也减轻了参与婴儿研究的家庭的负担。考虑到这些问题，我们将开发下一代方法来评估婴儿情绪调节和婴儿与父母的互动。为此，我们将使用 LittleBeats是我们团队开发的一款婴儿多模式可穿戴设备，可在家中长时间（每天约8-10小时）收集婴儿和父母发声（通过麦克风）、婴儿运动活动（通过运动传感器）和婴儿心脏迷走神经张力（通过心电图[ECG]）的时间同步数据。我们提出三个具体目标。首先，我们将验证黄金标准的静止面孔范式的虚拟访问协议，这通常是在实验室环境中进行的，用于评估婴儿在生命的第一年的情绪调节。其次，我们将验证多模态深度学习算法，以真实的时间检测婴儿的情绪状态 使用LittleBeats音频、ECG和运动数据。第三，我们将验证深度学习算法，以检测和标记婴儿（牙牙学语，大惊小怪，哭，笑）和父母（婴儿直接讲话，成人直接讲话，唱歌，笑）的发声类型，这些类型创建了婴儿-父母声音互动的构建块，例如轮流。通过将创新的可穿戴技术与尖端的深度学习算法结合在一起，我们的目标是进一步了解产前物质暴露导致不良后果的机制。此外，产前物质暴露是一种异质现象，与环境风险和 保护因素，从而使一刀切的做法无效。通过监测婴儿情绪调节的每时每刻的变化，结合深度学习算法，在婴儿表现出痛苦迹象的时刻检测和分类婴儿与父母的互动，所提出的方法有可能改变我们对动态过程的理解产前物质暴露导致不良结果并查明促进最佳发育的保护因素。