Brain-inspired non-stationary learning.

计算能力和内存存储大约每两年翻一番，使今天的计算机基本上可以记住一切。与此同时，正在开发新的机器学习技术，利用这些丰富的数据来提取知识，进行预测，并推广到看不见的数据;其中许多以人工神经网络为核心。这种结合为许多真实的世界问题带来了令人印象深刻的新解决方案，包括图像分类和语音处理。尽管取得了这些进展，但计算机在更通用的任务上仍然落后于人类。特别是，目前的方法不太适合在非平稳环境中学习（数据随时间变化）：理想的系统应该快速学习新事物，而不会忘记以前知道的东西。为了阐明这些想法，考虑一个经过训练的人工神经网络，用于对图像中的衣服进行分类。这是一个非静态的任务，因为时尚在变化和创新，所以网络必须不断地从新的例子中学习。然而，它必须在不忘记以前的例子的情况下这样做（例如，夏天的衣服，整个冬天都看不到），否则它将不得不在每年春天从头开始重新学习夏天的衣服。在实践中，为了处理新的例子，网络需要以很高的速度学习，但这种高学习速度有一个副作用，即系统会很快忘记。相反，如果学习率低，网络的记忆时间会长得多，但学习速度会慢得不切实际，无法灵活应对不断变化的环境。因此，在不遗忘的情况下快速学习非静态任务是一个基本的研究挑战，被认为是当前迁移学习、持续学习或终身学习方法的绊脚石。但当然，有一个系统已经解决了这个明显的困境：人脑。我们人类生活在一个非静止的世界中，我们既可以快速学习，又可以长时间记忆。实验心理学的一个经典例子表明，一个人忘记一系列先前记忆的随机字母的速度遵循幂律，即，在1小时和2小时之间的衰减与在2小时和4小时之间或在1周和2周之间的衰减一样大。相比之下，人工系统中的遗忘呈指数级发生，即，在1h和2 h之间的衰减与在100 h和101 h之间的衰减相同，因此比在人类中观察到的要快得多。在大脑中，学习是基于当新模式进入时神经元之间的连接强度的修改，这一过程称为突触可塑性。这种变化可以持续不同的时间，从而产生三个时间尺度：短期可塑性，长期可塑性和突触巩固。本提案的研究假设是，我们可以通过构建一个学习系统来达到人类水平的表现，该学习系统从大脑的这些学习机制中获得灵感，特别是突触可塑性的不同时间尺度及其相互作用。直觉是这样的：一个传入的记忆是用最快的学习速率快速学习的，然后这个记忆被慢慢地转移到另一个以较慢的学习速率运行的组件上，这样它就不会被新传入的记忆覆盖。因此，这项建议解决了两个研究挑战。我打算在所有三个学习时间尺度上建立一个统一的学习规则，就像我在过去的工作中统一长期和非常长期一样。然后，我将研究塑料网络的学习和遗忘速度与统一的学习规则。该网络将学习对非平稳数据进行分类，但要对所有可见数据进行测试，这在机器学习中是一项非常困难的任务。

Computing power and memory storage have doubled approximately every two years, allowing today's computers to memorise essentially everything. In tandem, new machine learning techniques are being developed that harness this wealth of data to extract knowledge, make predictions, and generalize to unseen data; many of these with artificial neural networks at their core. This combination has led to impressive new solutions to numerous real world problems, including image classification and speech processing. Despite this progress, computers still lag behind human performance on more general-purpose tasks. In particular, current methods are not well suited to learning in non-stationary settings (where the data is changing over time): a desirable system would learn new things quickly, without forgetting what it knew before. To clarify these ideas, consider an artificial neural network trained to classify clothes from images. This is a non-stationary task, because fashions change and innovate, so the network must continually learn from new examples. However, it must do so without forgetting previous examples (e.g. summer clothes, not seen for all of winter), otherwise it would have to relearn about summer clothes from scratch each spring. In practice, to handle new examples, the network needs to learn at a high rate, but this high learning rate has the side-effect of overwriting old memories; that is, the system is forgetting quickly. Conversely, if the learning rate is low, the network remembers for much longer, but then learning is impractically slow, and no longer agile enough to deal with changing environments.This research challenge of fast learning on non-stationary tasks without forgetting is therefore a fundamental one, and is recognized as a stumbling block in current approaches to transfer learning, continual learning or life-long learning. But of course, there exists one system that has solved the apparent dilemma: the human brain. We humans live our life in a non-stationary world, and we can both learn quickly and remember for a long time. A classical example from experimental psychology shows that the rate at which a person forgets a series of previously memorised random letters follows a power-law, i.e., the decay is equally large between 1h and 2h as it is between 2h and 4h, or between 1 week and 2 weeks. In contrast, forgetting in artificial systems happens exponentially, i.e., the decay is the same between 1h and 2h as it is between 100h and 101h, and therefore much faster than observed in humans.In the brain, learning is based on the modification of the connection strength between neurons when a new pattern enters, a process called synaptic plasticity. This change can last for different amounts of time, giving rise to the three timescales: short-term plasticity, long-term plasticity and synaptic consolidation.The research hypothesis of this proposal is that we can reach human-level performance by building a learning system that takes inspiration from these learning mechanisms of the brain, in particular the different time scales of synaptic plasticity and their interplay. The intuition is the following: an incoming memory is learnt quickly using the fastest learning rate, then this memory is slowly transferred to another component that operates at a slower learning rate, so that it is not overwritten by new incoming memories. This proposal therefore addresses two research challenges. I intend to build a unifying learning rule across all three learning timescales, just like I unified long-term and very long-term in past work. I will then investigate the learning and forgetting speed in plastic networks with the unifying learning rule. The network will learn to categorise on non-stationary data, but be tested on all the seen data, currently a very difficult task in machine learning.