Unsupervised learning for perception

点击翻译按钮获取中文摘要

The backpropagation learning procedure that is often used to train artificial neural networks does notwork well for deep networks that have many layers of non-linear neurons between the input and theoutput.  My group has recently introduced a new approach in which deep networks are "pretrained" onelayer at a time using an efficient unsupervised algorithm that tries to model regularities in theinput data. After pretraining, the intermediate layers of a deep network already contain many usefulfeature detectors and backpropagation is much better at fine-tuning a pretrained deep network than itis at learning a deep network from scratch.  In a recent Science paper we showed that this combinationof unsupervised pretraining followed by supervised fine-tuning outperforms all other learning methodsat a well-studied discrimination task and also for non-linear dimensionality reduction.Using this new approach, we plan to apply deep neural networks to a variety of important discrimintiontasks including object and face recognition. We will also use deep networks to model human motion,speech, and video sequences.  Using deep networks to extract low-dimensional codes from large, highdimensional datasets, we will improve web-based recommendation systems. We will also develop deepnetworks that can convert documents or images into binary addresses in such a way that semanticallysimilar documents have similar binary addresses. This allows extremely fast search for similardocuments or images. Finally, we will investigate ways in which these new learning algorithms might beimplemented in the cortex.