Brain--inspired disinhihbitory learning rule for continual learning tasks in artificial neural networks

机器在学习任务（如图像分类或语音识别）方面的表现接近人类，但大多数最先进的解决方案在固定环境中表现出色。另一方面，部署在现实场景中的系统需要能够在不断变化的环境中学习。为什么这是一个挑战？每当一个典型的学习系统遇到一个新任务时，它就会用它在新任务上学到的东西来覆盖以前任务的解决方案。想象一下，一个用于老年人护理的机器人：经过两个月的训练，它可以载着人上下楼梯，房子里有装修，机器人学会了用临时升降机运送人。如果机器人因此忘记了它在楼梯上的导航技能，并且在翻新后的两个月内不得不重新学习楼梯的状况，那将是愚蠢的。目前最先进的机器学习算法有这样的局限性，一个叫做持续学习或终身学习的挑战。对于这个EPSRC奖学金，我们计划开发一个大脑启发的学习算法，并在人工神经网络中测试它，解决一个持续学习的任务。让我们来看看大脑是如何解决持续学习的。人类具有适应环境和记忆经验的迷人能力;两者都需要记忆。我们可以学得很快，记住很长时间，但这导致了一个困境：为了快速学习，大脑需要非常容易地改变，即是塑料，但为了记住很长一段时间，大脑不能太可塑。神经水平上学习和记忆的基础是神经元之间连接的变化，称为突触可塑性。科学家们已经对突触可塑性进行了彻底的研究，主要集中在兴奋性神经元上，而大多忽视了抑制性神经元的作用。我们认为，与其在所有经验中平等地学习，解决困境的办法是调节哪些记忆要学习，从而避免不必要的重要记忆的混淆。我们认为抑制是调节学习的关键，因为降低抑制为学习打开了一扇大门。为了验证这一假设，在EPSRC奖学金中，我们将使用递归网络中的计算模型来研究兴奋性和抑制性可塑性的相互作用。然后，我们将测试抑制是否以及如何控制突触可塑性，从而学习。最后，我们将在强化学习框架下的导航持续学习任务中测试我们的大脑启发学习规则的性能。

Machines are achieving near-human performance at learning tasks such as image categorisation or speech recognition, but most of the state-of-the-art solutions excel in fixed environments. Systems deployed in real-world scenario, on the other hand, need to be able to learn in changing environments. Why is this a challenge? Every time a typical learning system encounters a new task, it overwrites the solution to previous tasks by what it learns on the new one. Imagine a robot used for elderly care: After two months of training it to carry the person up and down the stairs, there are renovations in the house, and the robot learns to transport the person with a temporary lift. It would be silly if the robot would thereby unlearn its skills for navigating stairs and would have to relearn the stair condition for two months after the renovation. Current state-of-the art machine learning algorithms have this limitation, a challenge called continual learning, or life-long learning.For this EPSRC Fellowship, we plan to develop a brain-inspired learning algorithm and test it in artificial neural networks solving a continual learning task. So, let's look at how the brain might solve continual learning. Humans have the fascinating ability to adapt to their environment and memorise experiences; both require memory. We can learn quickly and remember for a long time, but this leads to a dilemma: In order to learn quickly, the brain needs to change very easily i.e. be plastic, but in order to remember for a long time, the brain must not be too plastic. The basis of learning and memory at the neural level are changes in the connections between neurons, called synaptic plasticity. Scientists have worked thoroughly to characterise synaptic plasticity, focusing on excitatory neurons, while mostly neglecting the role of inhibitory neurons. We suggest that instead of learning equally across all experience, the solution to the dilemma is to regulate which memories to learn, therefore avoiding unnecessary overwriting of important memories. We propose that inhibition is the key to regulating learning, in that lowering inhibition opens a gate for learning. To test this hypothesis, in this EPSRC Fellowship we will investigate the interaction of excitatory and inhibitory plasticity using computational models in recurrent networks. We will then test whether and how inhibition gates synaptic plasticity and therefore learning. Finally, we will test the performance of our brain-inspired learning rule in a continual learning task of navigation under a reinforcement learning framework.