Brain--inspired disinhihbitory learning rule for continual learning tasks in artificial neural networks

机器在图像分类或语音识别等学习任务中取得了接近人类的表现，但大多数最先进的解决方案在固定环境中表现出色。另一方面，在真实场景中部署的系统需要能够在不断变化的环境中学习。为什么这是一个挑战？每次典型的学习系统遇到新任务时，它都会用在新任务上学到的知识覆盖先前任务的解决方案。想象一下一个用于老年护理的机器人：经过两个月的训练，它可以抬着人上下楼梯，房子里有翻新，机器人学会用临时升降机运送人。如果机器人因此失去了导航楼梯的技能，并在翻新后两个月内不得不重新学习楼梯状况，那将是愚蠢的。当前最先进的机器学习算法存在这种局限性，这是一种称为持续学习或终身学习的挑战。为了这次EPSRC联谊会，我们计划开发一种受大脑启发的学习算法，并在解决连续学习任务的人工神经网络中对其进行测试。那么，让我们来看看大脑是如何解决持续学习的。人类具有令人着迷的适应环境和记忆经验的能力；两者都需要记忆。我们可以快速学习并长时间记忆，但这导致了一个两难境地：为了快速学习，大脑需要非常容易地改变，即变得可塑性，但为了长时间记忆，大脑不能太可塑性。在神经层面上学习和记忆的基础是神经元之间连接的变化，称为突触可塑性。科学家们对突触的可塑性进行了彻底的研究，重点放在兴奋性神经元上，而大多忽略了抑制性神经元的作用。我们建议，解决困境的办法不是在所有经历中平等地学习，而是规范学习哪些记忆，从而避免不必要地覆盖重要记忆。我们认为抑制是调节学习的关键，因为降低抑制打开了学习的大门。为了验证这一假设，在这次EPSRC联谊会中，我们将使用递归网络中的计算模型来研究兴奋性和抑制性可塑性的相互作用。然后，我们将测试抑制是否以及如何限制突触的可塑性，从而影响学习。最后，我们将测试我们的大脑启发学习规则在强化学习框架下导航的持续学习任务中的性能。

Machines are achieving near-human performance at learning tasks such as image categorisation or speech recognition, but most of the state-of-the-art solutions excel in fixed environments. Systems deployed in real-world scenario, on the other hand, need to be able to learn in changing environments. Why is this a challenge? Every time a typical learning system encounters a new task, it overwrites the solution to previous tasks by what it learns on the new one. Imagine a robot used for elderly care: After two months of training it to carry the person up and down the stairs, there are renovations in the house, and the robot learns to transport the person with a temporary lift. It would be silly if the robot would thereby unlearn its skills for navigating stairs and would have to relearn the stair condition for two months after the renovation. Current state-of-the art machine learning algorithms have this limitation, a challenge called continual learning, or life-long learning.For this EPSRC Fellowship, we plan to develop a brain-inspired learning algorithm and test it in artificial neural networks solving a continual learning task. So, let's look at how the brain might solve continual learning. Humans have the fascinating ability to adapt to their environment and memorise experiences; both require memory. We can learn quickly and remember for a long time, but this leads to a dilemma: In order to learn quickly, the brain needs to change very easily i.e. be plastic, but in order to remember for a long time, the brain must not be too plastic. The basis of learning and memory at the neural level are changes in the connections between neurons, called synaptic plasticity. Scientists have worked thoroughly to characterise synaptic plasticity, focusing on excitatory neurons, while mostly neglecting the role of inhibitory neurons. We suggest that instead of learning equally across all experience, the solution to the dilemma is to regulate which memories to learn, therefore avoiding unnecessary overwriting of important memories. We propose that inhibition is the key to regulating learning, in that lowering inhibition opens a gate for learning. To test this hypothesis, in this EPSRC Fellowship we will investigate the interaction of excitatory and inhibitory plasticity using computational models in recurrent networks. We will then test whether and how inhibition gates synaptic plasticity and therefore learning. Finally, we will test the performance of our brain-inspired learning rule in a continual learning task of navigation under a reinforcement learning framework.