Neural dynamics underlying spatiotemporal cognitive integration

点击翻译按钮获取中文摘要

Project Summary Our ability to visually interpret the world around us depends on bottom-up computations that extract relevant information from the sensory inputs but it also depends on our accumulated core knowledge about the world providing top-down signals based on prior experience. The goal of this proposal is to study the mechanisms by which visual information is integrated spatially and temporally to combine bottom-up and top- down knowledge. Towards this goal, we combine behavioral measurements, invasive neurophysiological recordings, and computational models. The behavioral data will provide critical constraints about human integrative abilities, particularly through eye movements and the dynamics of recognition. The invasive neurophysiological data will provide high spatiotemporal resolution of neural activity along the inferior temporal cortex and the interactions with pre-frontal cortex, which is hypothesized to be critical for conveying the type of top-down signals required for recognition. Ultimately, a central goal of our proposal is to formalize our understanding of these integrative process via a quantitative computational model. This computational model should be able to capture the behavioral and physiological results and provide testable predictions. During the current award, we have made significant progress towards elucidating the mechanisms underlying pattern completion whereby the visual system is capable of inferring the identity of objects from partial information. Here we consider a set of images and videos that are “minimal” in the sense that they are recognizable but where any further reduction in the amount of spatial or temporal information renders them unrecognizable. We have strong preliminary evidence that suggests that state-of-the-art purely bottom-up theories of recognition instantiated by deep convolutional networks cannot explain human behavior and physiology. Therefore, these types of stimuli provide an ideal arena to investigate how top-down signals, presumably from pre-frontal cortex, modulate the responses along ventral visual cortex to orchestrate recognition. Understanding the neural mechanisms by which core knowledge is incorporated into sensory processing is arguably one of the greatest challenges in Cognitive Science and may have important implications for many neurological and psychiatric conditions that are characterized by dysfunctional top-down signaling and remain poorly understood.