Predictive-Coding Neural Networks: How They Learn and Behave

点击翻译按钮获取中文摘要

The brain is a bewilderingly complex organ, but progress is being made to unlock its secrets. The theory of Predictive Coding (PC) states that the brain is a hierarchy of layers in which each layer communicates only with the layer below and the layer above. This architecture mimics the reciprocal connectivity found between brain regions in the mammalian cortex. Information in PC networks flows both up and down the network simultaneously. Our brains do the same, suggesting that perception is an active, two-way inference process. We hypothesize that this two-way process makes perception more robust. A number of challenges remain for PC networks. Firstly, some of the beneficial properties of PC classification networks are thought to stem from them being generative. Instead of feeding an image of a "2" into the bottom of the network and having the network classify it as a two, one can also feed in the class vector for "two" at the top of the network and have the network generate an image of a two. Unfortunately, those images typically do not resemble the training data, but appear random. We will investigate a variety of techniques to encourage PC networks to generate recognizable inputs. Generative PC networks might be more robust at perception than artificial neural networks (ANNs). Current ANNs are easy to fool by adding slight adjustments to the input. Yet, these adversarial inputs do not fool humans. We will test our generative PC networks against adversarial inputs to see if they fall victim to the same attacks, or if they are closer to human perception. It was recently revealed that PC networks approximate the backpropagation learning algorithm; as they run, they learn autonomously, unlike ANNs that require an external governing process to manage the learning. This suggests that something akin to backprop could be going on in the brain. However, the link to backprop was proven only for a restricted, layered architecture. We will extend the theory to derive biologically plausible learning algorithms for other common network topologies, such as layers with recurrent connections. Such a layer could model an attractor network and implement the dynamic process of perception, including the SoftMax-like bistable process of looking at an optical illusion. The learning algorithms developed in this research program will revolutionize AI and inform cognitive neuroscience. These robust neural networks will make mission-critical applications (like vision systems for driverless cars) more reliable. Moreover, these learning methods are autonomous, so PC networks can learn without a supervising program and, as a result, would be easier to deploy in specialized hardware, like those being developed in the emerging industry of neuromorphic chips. The students involved in this research will have a strategic advantage in the fields of AI and neuroscience, and will be leaders in the industrial revolution and expand the theory and practice in academic research.