CRCNS Research Proposal: Predictive Coding Network for Human Vision

点击翻译按钮获取中文摘要

This project aims to advance scientific knowledge about human vision and use neuroscience to enhance artificial intelligence for computer vision. Vision is central to how humans see and explore the world. About a dozen brain regions work together to process visual information within a fraction of a second. It is hypothesized that these brain regions actively predict one's visual surroundings and use errors of prediction to update their internal representations and guide actions. However, it is not clear how the brain performs computations for recognition and prediction, and whether it is possible for a machine to mimic the brain and recognize and predict visual input in complex, noisy, and uncertain circumstances. This project will address these questions from computational, psychological, and neuroscientific perspectives and deliver new models, data, and tools that promote the synergy between artificial intelligence and neuroscience.Investigators will design a model based on predictive coding in the brain, and test its ability to perform computer vision tasks and explain human behaviors and brain responses to naturalistic visual stimuli. The investigators will first develop a deep neural network referred to as the predictive coding network. Unlike existing feedforward neural networks, the currently predominant vision models, the predictive coding network has several defining features relevant to neural processing in the brain. It is bi-directional, processing information both bottom-up and top-down. It is recurrent, utilizing the same architecture for dynamic computation. It is parallel, allowing information processing to occur in parallel both within and across different layers. It is both discriminative and generative, reconciling image recognition and synthesis in a single framework. The predictive coding network will be evaluated against benchmark data sets. It is hypothesized to reach competitive performance with many fewer parameters than the state of the art. Then, the investigators will test the model's behaviors given naturalistic images degraded in various ways and/or presented for various durations. It is hypothesized that the model will be more robust and accurate after running for increasingly longer times and reach a time-accuracy tradeoff like human perception under similar conditions. To test this hypothesis, the investigators will perform human behavioral experiments and compare the model's behaviors against human behaviors. Further, the investigators will test the model's ability to explain brain responses to naturalistic images and videos, measured with functional magnetic resonance imaging and intracranial electroencephalography. The model is hypothesized to be able to predict the brain's dynamic activity and representation given naturalistic stimuli. The successful completion of this project is expected to deliver a brain-inspired vision model learnable and computable end-to-end. This model will empower machines with adaptive and robust vision and provide a tool for understanding the computational basis of biological vision.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.