Individual differences in natural scene semantics investigated with machine learning and neuroimaging

点击翻译按钮获取中文摘要

The ability to make sense of our visual environment is crucial for adaptive behaviour and survival. Yet, precisely how this subjective, conscious experience of the world emerges in our brain remains poorly understood. The long term aim of my research programme is to provide a new way of thinking about the visual system, placing emphasis on semantic representations and individual differences in the context of natural scene processing. Doing so, we will fill an important gap in our understanding of visual recognition by elucidating the neural underpinnings of semantic processing and its relationship with visual consciousness and subjective experience. How does the brain transform visual information from the retina into high-level semantic representations enabling us to clearly and precisely communicate what we see? Classic accounts suggest functional segregation in two distinct cortical pathways: a "where" and a "what" pathway, a dorsal stream specialised in spatial information ("where") and a ventral stream for category or conceptual information. The ventral stream could therefore be seen as a distributed system where overlapping feature maps encode specific dimensions about particular objects. This view might stem from simple experimental paradigms involving single objects or simplified stimuli, but it is not clear whether this view accurately generalises to naturalistic viewing situations with complex scenes containing multiple objects and concepts. Could there be higher-level (linguistic and semantic conscious form) features, being integrated over object and scene feature maps to provide meaning? To answer these questions, we will combine brain activity measured from electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) while participants view rich and complex natural scenes, with artificial neural networks (ANNs) trained on sentence embeddings, and a battery of innovative behavioural tasks (collected using my online platform for behavioural experiments https://meadows-research.com). The focus for this DG cycle is on moving theories of visual representations into the domains of complex naturalistic stimuli, semantic representations, and individual differences, all of which are relatively unexplored territories in the field. This will be achieved with three core aims: 1- Precisely track where in the brain and when in time after viewing a natural scene, semantic representations are established. 2- Reveal how activity patterns related to scene semantics influence the content of our visual consciousness. 3- Identify links between fine-scale differences in brain representations (i.e. the variations in activity pattern discriminability between individuals) and behaviour (measured in an array of cognitive tasks).