Brain-inspired non-stationary learning.

计算能力和内存存储大约每两年翻一番，这使得今天的计算机基本上可以记忆一切。与此同时，正在开发新的机器学习技术，利用这些丰富的数据来提取知识、进行预测，并将其概括为看不见的数据；其中许多技术以人工神经网络为核心。这种结合已经为许多现实世界的问题带来了令人印象深刻的新解决方案，包括图像分类和语音处理。尽管取得了这些进步，但在更多通用任务上，计算机仍然落后于人类的表现。特别是，目前的方法不太适合在非静态环境下学习(数据会随着时间的推移而变化)：一个理想的系统将快速学习新事物，而不会忘记它以前知道的东西。为了阐明这些想法，考虑一下一个人工神经网络，它被训练成从图像中对衣服进行分类。这是一项不固定的任务，因为时尚在变化和创新，因此网络必须不断向新的榜样学习。然而，它必须在不忘记以前的例子的情况下这样做(例如，夏天的衣服，整个冬天都看不到)，否则它将不得不在每年春天从头开始学习夏天的衣服。在实践中，为了处理新的例子，网络需要以高速率学习，但这种高学习速率有覆盖旧记忆的副作用；也就是，系统正在迅速遗忘。相反，如果学习率较低，网络会记住更长的时间，但学习会变得不切实际地缓慢，不再足够灵活地应对不断变化的环境。因此，在非静态任务上快速学习而不会忘记的研究挑战是一个基本挑战，被认为是当前迁移学习、持续学习或终身学习方法的绊脚石。但当然，有一个系统已经解决了这个明显的困境：人脑。我们人类生活在一个不静止的世界里，我们既能快速学习，又能长久记忆。一个来自实验心理学的经典例子表明，一个人忘记一系列以前记忆的随机字母的速度遵循幂定律，即在1小时到2小时之间，衰减率与在2小时到4小时之间，或者在1周到2周之间一样大。相比之下，在人工系统中，遗忘以指数方式发生，即在1小时至2小时之间的衰退与100小时至101小时之间的衰退相同，因此比人类观察到的要快得多。在大脑中，学习是基于新模式进入时神经元之间连接强度的改变，这一过程被称为突触可塑性。这种变化可以持续不同的时间，产生三个时间尺度：短期可塑性、长期可塑性和突触巩固。这一提议的研究假设是，我们可以通过建立一个学习系统来达到人类水平的表现，该学习系统从大脑的这些学习机制中获得灵感，特别是突触可塑性的不同时间尺度及其相互作用。直觉是这样的：一个传入的记忆以最快的学习速度被快速学习，然后这个记忆被缓慢地转移到另一个以较慢的学习速度操作的组件，这样它就不会被新的传入记忆覆盖。因此，这项提议解决了两个研究挑战。我打算建立一个统一的学习规则，跨越所有三个学习时间表，就像我在过去的工作中统一了长期和非常长期的学习规则一样。然后，我将使用统一的学习规则来研究塑料网络中的学习和遗忘速度。该网络将学习对非平稳数据进行分类，但要对所有已见数据进行测试，这目前是机器学习中的一项非常困难的任务。

Computing power and memory storage have doubled approximately every two years, allowing today's computers to memorise essentially everything. In tandem, new machine learning techniques are being developed that harness this wealth of data to extract knowledge, make predictions, and generalize to unseen data; many of these with artificial neural networks at their core. This combination has led to impressive new solutions to numerous real world problems, including image classification and speech processing. Despite this progress, computers still lag behind human performance on more general-purpose tasks. In particular, current methods are not well suited to learning in non-stationary settings (where the data is changing over time): a desirable system would learn new things quickly, without forgetting what it knew before. To clarify these ideas, consider an artificial neural network trained to classify clothes from images. This is a non-stationary task, because fashions change and innovate, so the network must continually learn from new examples. However, it must do so without forgetting previous examples (e.g. summer clothes, not seen for all of winter), otherwise it would have to relearn about summer clothes from scratch each spring. In practice, to handle new examples, the network needs to learn at a high rate, but this high learning rate has the side-effect of overwriting old memories; that is, the system is forgetting quickly. Conversely, if the learning rate is low, the network remembers for much longer, but then learning is impractically slow, and no longer agile enough to deal with changing environments.This research challenge of fast learning on non-stationary tasks without forgetting is therefore a fundamental one, and is recognized as a stumbling block in current approaches to transfer learning, continual learning or life-long learning. But of course, there exists one system that has solved the apparent dilemma: the human brain. We humans live our life in a non-stationary world, and we can both learn quickly and remember for a long time. A classical example from experimental psychology shows that the rate at which a person forgets a series of previously memorised random letters follows a power-law, i.e., the decay is equally large between 1h and 2h as it is between 2h and 4h, or between 1 week and 2 weeks. In contrast, forgetting in artificial systems happens exponentially, i.e., the decay is the same between 1h and 2h as it is between 100h and 101h, and therefore much faster than observed in humans.In the brain, learning is based on the modification of the connection strength between neurons when a new pattern enters, a process called synaptic plasticity. This change can last for different amounts of time, giving rise to the three timescales: short-term plasticity, long-term plasticity and synaptic consolidation.The research hypothesis of this proposal is that we can reach human-level performance by building a learning system that takes inspiration from these learning mechanisms of the brain, in particular the different time scales of synaptic plasticity and their interplay. The intuition is the following: an incoming memory is learnt quickly using the fastest learning rate, then this memory is slowly transferred to another component that operates at a slower learning rate, so that it is not overwritten by new incoming memories. This proposal therefore addresses two research challenges. I intend to build a unifying learning rule across all three learning timescales, just like I unified long-term and very long-term in past work. I will then investigate the learning and forgetting speed in plastic networks with the unifying learning rule. The network will learn to categorise on non-stationary data, but be tested on all the seen data, currently a very difficult task in machine learning.