Correlation and agreement between Language ENvironment Analysis (lena) and manual transcription for Dutch natural language recordings
Correlation and agreement between Language ENvironment Analysis (lena) and manual transcription for Dutch natural language recordings
复制标题
DOI：
10.3758/s13428-017-0960-0
复制
发表时间：
2018-10-01
影响因子：
5.4
通讯作者：
van Wieringen, Astrid
中科院分区：
文献类型：
--
作者：
Busch, Tobias;Sangen, Anouk;van Wieringen, Astrid
语言环境分析系统(Lena)自动分析儿童的自然声音环境。在其他方面，它估计了儿童接触到的成人词汇(Awc)、儿童发声(Cv)、对话(Ct)和电子媒体(Tv)的数量。为了评估Lena的可靠性，我们将其与手动转录进行了比较。具体地说，我们计算了48个5分钟音频样本的Lena估计与手动计数之间的相关性和一致性。这些样本是从6名荷兰语儿童(2-5岁)长达8天的录音中挑选出来的。AWC和Cv的相关性很强，r=0.87和r=0.77，而ct和Tv的相关性相对较低，r=0.52，r=0.50。然而，一致性分析显示，AWC计数存在恒定的偏差，cv和ct存在比例偏差(即偏差随cv和ct的值而变化)。关于检测电子媒体的协议很差。此外，所有四个指标的一致性范围都很宽。也就是说，Lena和单个音频样本的手工转录之间的差异在平均差异附近变化很大。这种变异可能表明莉娜受到了样本之间的差异的影响，而这些差异对人类转录者的影响并不相同。这些分歧和偏见使人们对不同家庭和时间的Lena测量的可比性产生了怀疑，这对在研究中使用Lena至关重要。我们的样本太小，无法得出Lena的测量在什么范围内是可比较的，但似乎明智的做法是谨慎对待可能系统性地偏向Lena的表现从而造成混乱的因素。
The Language ENvironment Analysis system (lena) automatically analyzes the natural sound environments of children. Among other things, it estimates the amounts of adult words (awc), child vocalizations (cv), conversational turns (ct), and electronic media (tv) that a child is exposed to. To assess lena's reliability, we compared it to manual transcription. Specifically, we calculated the correlation and agreement between the lena estimates and manual counts for 48 five-min audio samples. These samples were selected from eight day-long recordings of six Dutch-speaking children (ages 2-5). The correlations were strong for awc, r=.87, and cv, r=.77, and comparatively low for ct, r=.52, and tv, r=.50. However, the agreement analysis revealed a constant bias in awc counts, and proportional biases for cv and ct (i.e., the bias varied with the values for cv and ct). Agreement for detecting electronic media was poor. Moreover, the limits of agreement were wide for all four metrics. That is, the differences between lena and the manual transcriptions for individual audio samples varied widely around the mean difference. This variation could indicate that lena was affected by differences between the samples that did not equally affect the human transcribers. The disagreements and biases cast doubt on the comparability of lena measurements across families and time, which is crucial for using lena in research. Our sample is too small to conclude within which limits lena's measurements are comparable, but it seems advisable to be cautious of factors that could systematically bias lena's performance and thereby create confounds.