Using eye-tracking and in-depth video analysis to understand how speech input relates to speech development

使用眼动追踪和深入的视频分析来了解语音输入与语音发展的关系

• 批准号: RTI-2021-00487
• 负责人: Johnson, Elizabeth
• 金额: $ 8.6万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Research Tools and Instruments
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=718686
• 关键词: Using; eye; tracking; depth; video

Fifty years ago, the field of infant speech perception was born. In a study that revolutionized the way we understand early cognitive development, infants were shown to perceive speech sounds in a categorical manner much like adults do. Subsequent work demonstrated that this behaviour was universal, and independent of language experience. Since these initial foundational studies, experimental paradigms designed to measure children's attentional shifts in response to speech sounds have revealed that the development of spoken language depends on an intricate interplay between innate knowledge and experience. But until recently, findings have been primarily based on data that has been hand-coded by experimenters, an approach susceptible to experimenter bias, and unable to capture the subtle, fine-grained behaviour that often is necessary to answer nuanced experimental questions. And most analyses have collapsed observations across dozens of infants, focusing on group-level effects and therefore unable to address the question of how individual experiences are linked to individual differences in speech development. Thankfully, recent technological advancements offer more sophisticated methods to test questions about infant development. Eye-trackers can automatically record children's gaze/pupil dilation as the speech signal unfolds, and specialized software can tailor experiments in real time in response to children's performance. Highly engaging experimental stimuli can be created using advanced animation technology, improving the quality of data collected in the lab. And improvements in audio-visual technology and digital memory make it feasible to collect data-rich longitudinal recordings of individual caregiver-child interactions, which can then be aligned with individual performance in perception studies. Johnson has been a trailblazer in the use of these technologies to study speech development (e.g., she developed the first large-scale video corpus of caregiver-infant interactions linked to eyetracking data, and worked with SR Research to help them beta-test their first arm mount for their infant eyetracking system). But the equipment Johnson purchased when she was first built her lab at U of Toronto is now becoming obsolete. To maintain her cutting-edge research program on speech development in linguistically-diverse communities, she will purchase a new Eyelink eyetracker and update the stimulus creation and audio-visual recording equipment in her lab. She will also replace outdated software and purchase a new type of headmounted eyetracker. This new infant speech testing setup will be widely shared with other cognitive scientists, most notably Assistant Professor Jessamyn Schertz an expert in the relationship between speech perception and production. Updating this equipment will contribute to Canada's training and research capacities, as well as advance our understanding of speech development in our increasingly multilingual population.

50年前，婴儿言语感知领域诞生了。在一项彻底改变了我们理解早期认知发展方式的研究中，婴儿被证明可以像成年人一样以分类的方式感知语音。随后的工作表明，这种行为是普遍的，独立于语言经验。自从这些最初的基础研究以来，旨在测量儿童对语音的注意力转移的实验范式已经揭示了口语的发展取决于先天知识和经验之间错综复杂的相互作用。但直到最近，研究结果主要是基于实验者手工编码的数据，这种方法容易受到实验者偏见的影响，无法捕捉到微妙的，细粒度的行为，而这些行为往往是回答微妙的实验问题所必需的。大多数分析都忽略了对几十个婴儿的观察，集中在群体水平的影响上，因此无法解决个体经验如何与言语发展中的个体差异联系起来的问题。值得庆幸的是，最近的技术进步提供了更复杂的方法来测试有关婴儿发育的问题。眼动追踪器可以自动记录儿童的凝视/瞳孔扩张，因为语音信号展开，和专门的软件可以调整实验在真实的时间，以响应儿童的表现。使用先进的动画技术可以创建高度吸引人的实验刺激，提高实验室收集的数据质量。视听技术和数字记忆的改进使得收集个体母子互动的数据丰富的纵向记录成为可能，然后可以与感知研究中的个体表现保持一致。约翰逊一直是使用这些技术研究语音发展的先驱者（例如，她开发了第一个与眼动追踪数据相关的婴儿与婴儿互动的大规模视频语料库，并与SR Research合作，帮助他们测试他们的婴儿眼动追踪系统的第一个手臂支架。但是约翰逊在多伦多大学第一次建立实验室时购买的设备现在已经过时了。为了在语言多样化的社区中保持她在语言发展方面的尖端研究计划，她将购买一台新的Eyelink眼动仪，并更新她实验室的刺激创造和视听记录设备。她还将更换过时的软件，并购买一种新型的头戴式眼动仪。这种新的婴儿语音测试设置将与其他认知科学家广泛分享，其中最著名的是助理教授Schertz，他是语音感知和产生之间关系的专家。更新这一设备将有助于加拿大的培训和研究能力，并增进我们对我们越来越多的多语种人口的语言发展的了解。