Credit assignment in the neocortex

当我们学习新东西时，我们的大脑会发生变化。具体来说，我们大脑中细胞之间的联系发生了变化。这些被称为突触的连接决定了信息如何在我们的大脑中流动，所以改变它们会改变我们的神经系统处理信息的方式。然而，值得注意的是，当我们学习时，我们不仅会改变，还会变得更好。尽管神经科学家现在对大脑中的突触变化有了一些了解，但确保我们在学习时确实在某件事上做得更好的机制仍然非常神秘。这个谜团被称为信用分配问题。不知何故，我们的大脑能够将任何错误或成功“归功于”构成行为基础的突触。换句话说，我们的大脑知道哪些突触需要改变，以减少我们的错误，增加我们的成功。有趣的是，被称为人工神经网络的模拟大脑使人工智能（AI）领域的研究人员能够构建能够识别图像、翻译语言、驾驶汽车和控制机械臂的系统。与真正的大脑一样，人工神经网络在学习时也会改变它们的突触连接。这是可行的，因为计算机科学家已经开发出了在人工神经网络中分配信用的算法。然而，人工神经网络仍然不能像人类一样学习。例如，一个人可以在几个小时内学会开车，而目前的人工神经网络需要数百万小时。因此，尽管人工神经网络中使用的信用分配算法对人工智能很重要，但它们并不能确切地告诉我们我们自己的大脑是如何工作的。在这个研究项目中，我们想要回答的问题是：哺乳动物的大脑是如何解决信用分配问题的，我们能否建立人工神经网络来模仿大脑的解决方案？我们将利用人工智能和神经科学的结合来解决这个问题。这项研究有很大的潜力造福社会。如果我们能理解大脑是如何解决信用分配问题的，那么我们就能同时实现两个重大飞跃。首先，我们将了解大脑在学习时是如何自我重组的。这可以帮助我们开发新的脑机接口、假肢设备等。其次，如果我们理解了大脑中的信用分配，我们就有可能为人工神经网络开发新的算法，使它们能够像人类一样学习。这可以使他们更快地学习。加拿大在神经科学启发的人工智能领域处于世界领先地位，这项研究将有助于培养下一代研究人员、软件工程师和企业家，帮助巩固我们在这一领域的地位，并为加拿大社会带来经济效益。

When we learn something new our brain changes. Specifically, the connections between the cells in our brains change. These connections, called synapses, determine how information flows through our brains, so changing them alters how our nervous system processes information. What is notable about learning, though, is that we don't just change when we learn, we get better. Though neuroscientists now have some understanding about synaptic changes in the brain, the mechanisms that ensure that we actually get better at something when we learn remain very mysterious. This mystery is known as the credit assignment problem. Somehow, our brains are capable of "assigning credit" for any errors or successes to the synapses that underlie a behaviour. Put another way, our brain somehow knows which synapses need to change in order to reduce our errors and increase our successes. Interestingly, simulated brains, known as artificial neural networks, have allowed researchers in the field of artificial intelligence (AI) to build systems that can recognise images, translate languages, drive cars, and control robotic arms. As with real brains, artificial neural networks change their synaptic connections when they learn. This works because computer scientists have developed algorithms for assigning credit in artificial neural networks. However, artificial neural networks still cannot learn as well as human beings. For example, a human can learn to drive a car in a matter of hours, whereas current artificial neural networks take millions of hours. Therefore, though the credit assignment algorithms used in artificial neural networks have been important for AI, they do not tell us exactly how our own brains work. The question we want to answer in this research program is: how does the mammalian brain solve the credit assignment problem, and can we build artificial neural networks that mimic the brain's solution? We will utilise a combination of AI and neuroscience to address this question. This research has significant potential to benefit society. If we could understand how the brain solves the credit assignment problem, then we could simultaneously make two significant leaps forward. First, we would understand how the brain rewires itself when learning. This could help us to develop novel brain-computer interfaces, prosthetic devices, etc. Second, if we understood credit assignment in the brain, we could potentially develop new algorithms for artificial neural networks that would give them the ability to learn more like humans. This could make them faster at learning. Canada is a world-leader in neuroscience inspired AI, and this research will help to train the next generation of researchers, software engineers, and entrepreneurs to help consolidate our position in this field and generate economic benefits for Canadian society.