iCatcher+: Robust and Automated Annotation of Infants' and Young Children's Gaze Behavior From Videos Collected in Laboratory, Field, and Online Studies.
iCatcher+: Robust and Automated Annotation of Infants' and Young Children's Gaze Behavior From Videos Collected in Laboratory, Field, and Online Studies.
复制标题
DOI：
10.1177/25152459221147250
复制
发表时间：
2023-04
影响因子：
13.6
通讯作者：
Liu, Shari
中科院分区：
文献类型：
--
作者：
Erel, Yotam;Shannon, Katherine Adams;Chu, Junyi;Scott, Kim;Struhl, Melissa Kline;Cao, Peng;Tan, Xincheng;Hart, Peter;Raz, Gal;Piccolo, Sabrina;Mei, Catherine;Potter, Christine;Jaffe-Dax, Sagi;Lew-Williams, Casey;Tenenbaum, Joshua;Fairchild, Katherine;Bermano, Amit;Liu, Shari
心理学研究的技术进步使人类行为的大规模研究成为可能，并简化了数据自动处理的流程。然而，对婴儿和儿童的研究并没有完全获得这些好处，因为即使这些数据是在线收集的，他们感兴趣的行为，如凝视时间和方向，仍然必须通过费力的手动注释过程从视频中提取。计算机视觉的最新进展提高了对这些视频数据进行自动注释的可能性。在本文中，我们通过工程改进，建立了一个用于幼儿自动注视标注的系统iCatcher，然后对该系统进行训练和测试（以下简称为iCatcher+）在具有大量视频和参与者可变性的三个数据集上进行（在美国实验室和现场收集的214个视频，在塞内加尔现场收集的143个视频，以及通过家庭网络摄像头收集的265个视频;参与者年龄范围= 4个月-3.5岁）。当在这些数据集上进行训练时，iCatcher+在所有数据集上区分“左”与“右”以及“开”与“关”的行为时，在展示的视频上表现出接近人类水平的准确性。这种高性能是在单个帧、实验性试验和研究视频的水平上实现的;在参与者人口统计学（例如，年龄、种族/民族），参与者行为（例如，移动，头部位置），和视频特性（例如，亮度）;并推广到第四个完全保留的在线数据集。最后，我们讨论了在线婴儿和儿童行为研究生命周期完全自动化所需的后续步骤，这是实现强大和高通量发展研究的关键一步。
Technological advances in psychological research have enabled large-scale studies of human behavior and streamlined pipelines for automatic processing of data. However, studies of infants and children have not fully reaped these benefits because the behaviors of interest, such as gaze duration and direction, still have to be extracted from video through a laborious process of manual annotation, even when these data are collected online. Recent advances in computer vision raise the possibility of automated annotation of these video data. In this article, we built on a system for automatic gaze annotation in young children, iCatcher, by engineering improvements and then training and testing the system (referred to hereafter as iCatcher+) on three data sets with substantial video and participant variability (214 videos collected in U.S. lab and field sites, 143 videos collected in Senegal field sites, and 265 videos collected via webcams in homes; participant age range = 4 months–3.5 years). When trained on each of these data sets, iCatcher+ performed with near human-level accuracy on held-out videos on distinguishing “LEFT” versus “RIGHT” and “ON” versus “OFF” looking behavior across all data sets. This high performance was achieved at the level of individual frames, experimental trials, and study videos; held across participant demographics (e.g., age, race/ethnicity), participant behavior (e.g., movement, head position), and video characteristics (e.g., luminance); and generalized to a fourth, entirely held-out online data set. We close by discussing next steps required to fully automate the life cycle of online infant and child behavioral studies, representing a key step toward enabling robust and high-throughput developmental research.
登录
查看更多内容
影响因子：
4.6
作者：
Fernald A;Marchman VA
通讯作者：
Marchman VA
DOI：
10.3758/bf03207207
发表时间：
1972-01-01
期刊：
PERCEPTION & PSYCHOPHYSICS
影响因子：
--
作者：
FRIEDMAN, S
通讯作者：
FRIEDMAN, S
影响因子：
8.2
作者：
Fernald, A;Pinto, JP;McRoberts, GW
通讯作者：
McRoberts, GW
影响因子：
2.6
作者：
Frank, Michael C.;Bergelson, Elika;Yurovsky, Daniel
通讯作者：
Yurovsky, Daniel
影响因子：
2.6
作者：
Erel, Yotam;Potter, Christine E.;Jaffe-Dax, Sagi;Lew-Williams, Casey;Bermano, Amit H.
通讯作者：
Bermano, Amit H.