Cerebellum-inspired parallel deep learning

受小脑启发的并行深度学习

基本信息

  • 批准号:
    EP/X029336/1
  • 负责人:
  • 金额:
    $ 52.6万
  • 依托单位:
  • 依托单位国家:
    英国
  • 项目类别:
    Research Grant
  • 财政年份:
    2024
  • 资助国家:
    英国
  • 起止时间:
    2024 至 无数据
  • 项目状态:
    未结题

项目摘要

Deep learning, a type of machine learning, has recently undergone dramatic developments. It is already having an impact across society, from scientific discovery to climate change prediction. However, current deep learning models require billions of parameters which can take several weeks to train, costing millions of pounds with large carbon footprints. It is therefore becoming increasingly important to develop efficient training methods for deep neural networks.One of the key bottlenecks that underlie the inefficiency of deep neural networks is the need to perform a large number of computational steps sequentially. Here, inspired by recent findings on how biological neural networks learn, we propose to develop an efficient training algorithm for deep neural networks. We have recently proposed that a specialised brain region, the cerebellum, enables parallel learning in the brain. The cerebellum is defined by two features that are well placed to facilitate parallel learning: sparsity and modularity. First, the cerebellum contains highly sparse connectivity with only four input connections per neuron, which should result in faster learning. Second, the cerebellum is a highly modular system, which is well placed to enable parallel learning. Inspired by these cerebellar features we will develop a sparse-modular system for training deep learning networks capable of efficient parallel training. In collaboration with industry, the benefits of this approach will be demonstrated using a new type of parallel processor designed to accelerate machine learning. Overall, our work will lead to a novel approach to parallel deep learning, leading to a substantial reduction in training times and costs.
深度学习是一种机器学习,最近经历了戏剧性的发展。它已经对整个社会产生了影响,从科学发现到气候变化预测。然而,目前的深度学习模型需要数十亿个参数,可能需要数周的时间来训练,成本高达数百万英镑,碳足迹巨大。因此,为深度神经网络开发高效的训练方法变得越来越重要。深度神经网络效率低下的关键瓶颈之一是需要顺序执行大量计算步骤。在这里,受最近关于生物神经网络如何学习的发现的启发,我们建议为深度神经网络开发一种有效的训练算法。我们最近提出,一个专门的大脑区域,小脑,使大脑中的平行学习。小脑有两个特征可以很好地促进并行学习:稀疏性和模块性。首先,小脑包含高度稀疏的连接,每个神经元只有四个输入连接,这应该会导致更快的学习。第二,小脑是一个高度模块化的系统,能够很好地实现并行学习。受这些小脑特征的启发,我们将开发一个稀疏模块化系统,用于训练能够进行高效并行训练的深度学习网络。在与工业界的合作中,这种方法的好处将通过一种旨在加速机器学习的新型并行处理器来展示。总的来说,我们的工作将为并行深度学习带来一种新的方法,从而大幅减少训练时间和成本。

项目成果

期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Rui Ponte Costa其他文献

Cerebellar-driven cortical dynamics can enable task acquisition, switching and consolidation
小脑驱动的皮质动力学能够实现任务获取、转换和巩固。
  • DOI:
    10.1038/s41467-024-55315-6
  • 发表时间:
    2024-12-30
  • 期刊:
  • 影响因子:
    15.700
  • 作者:
    Joseph Pemberton;Paul Chadderton;Rui Ponte Costa
  • 通讯作者:
    Rui Ponte Costa
Self-supervised predictive learning accounts for cortical layer-specificity
自我监督的预测性学习解释了皮质层特异性
  • DOI:
    10.1038/s41467-025-61399-5
  • 发表时间:
    2025-07-04
  • 期刊:
  • 影响因子:
    15.700
  • 作者:
    Kevin Kermani Nejad;Paul Anastasiades;Loreen Hertäg;Rui Ponte Costa
  • 通讯作者:
    Rui Ponte Costa

Rui Ponte Costa的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Rui Ponte Costa', 18)}}的其他基金

AI-driven modelling for cortex-wide neuromodulated learning
用于全皮层神经调节学习的人工智能驱动建模
  • 批准号:
    BB/X013340/1
  • 财政年份:
    2023
  • 资助金额:
    $ 52.6万
  • 项目类别:
    Research Grant
Dopaminergic-cholinergic neuromodulation for rapid and democratic cortex-wide learning
多巴胺能胆碱能神经调节用于快速和民主的皮质范围学习
  • 批准号:
    EP/Y027841/1
  • 财政年份:
    2023
  • 资助金额:
    $ 52.6万
  • 项目类别:
    Research Grant
AI-driven brain modelling for personalised cognitive enhancement
人工智能驱动的大脑建模,用于个性化认知增强
  • 批准号:
    MR/X006107/1
  • 财政年份:
    2022
  • 资助金额:
    $ 52.6万
  • 项目类别:
    Research Grant

相似国自然基金

多层次纳米叠层块体复合材料的仿生设计、制备及宽温域增韧研究
  • 批准号:
    51973054
  • 批准年份:
    2019
  • 资助金额:
    60.0 万元
  • 项目类别:
    面上项目

相似海外基金

CAREER: Origami-inspired design for a tissue engineered heart valve
职业:受折纸启发的组织工程心脏瓣膜设计
  • 批准号:
    2337540
  • 财政年份:
    2024
  • 资助金额:
    $ 52.6万
  • 项目类别:
    Continuing Grant
Convergence Accelerator Track M: Bio-Inspired Design of Robot Hands for Use-Driven Dexterity
融合加速器轨道 M:机器人手的仿生设计,实现使用驱动的灵活性
  • 批准号:
    2344109
  • 财政年份:
    2024
  • 资助金额:
    $ 52.6万
  • 项目类别:
    Standard Grant
BAMBOO - Build scAled Modular Bamboo-inspired Offshore sOlar systems
BAMBOO - 构建规模化模块化竹子式海上太阳能系统
  • 批准号:
    10109981
  • 财政年份:
    2024
  • 资助金额:
    $ 52.6万
  • 项目类别:
    EU-Funded
CAREER: Scalable Physics-Inspired Ising Computing for Combinatorial Optimizations
职业:用于组合优化的可扩展物理启发伊辛计算
  • 批准号:
    2340453
  • 财政年份:
    2024
  • 资助金额:
    $ 52.6万
  • 项目类别:
    Continuing Grant
CAREER: SHF: Bio-Inspired Microsystems for Energy-Efficient Real-Time Sensing, Decision, and Adaptation
职业:SHF:用于节能实时传感、决策和适应的仿生微系统
  • 批准号:
    2340799
  • 财政年份:
    2024
  • 资助金额:
    $ 52.6万
  • 项目类别:
    Continuing Grant
NSF-NSERC: Fairness Fundamentals: Geometry-inspired Algorithms and Long-term Implications
NSF-NSERC:公平基础:几何启发的算法和长期影响
  • 批准号:
    2342253
  • 财政年份:
    2024
  • 资助金额:
    $ 52.6万
  • 项目类别:
    Standard Grant
NSF Convergence Accelerator Track L: Intelligent Nature-inspired Olfactory Sensors Engineered to Sniff (iNOSES)
NSF 融合加速器轨道 L:受自然启发的智能嗅觉传感器,专为嗅探而设计 (iNOSES)
  • 批准号:
    2344256
  • 财政年份:
    2024
  • 资助金额:
    $ 52.6万
  • 项目类别:
    Standard Grant
Development of Integrated Quantum Inspired Algorithms for Shapley Value based Fast and Interpretable Feature Subset Selection
基于 Shapley 值的快速且可解释的特征子集选择的集成量子启发算法的开发
  • 批准号:
    24K15089
  • 财政年份:
    2024
  • 资助金额:
    $ 52.6万
  • 项目类别:
    Grant-in-Aid for Scientific Research (C)
NSF Convergence Accelerator Track M: Bio-Inspired Surface Design for High Performance Mechanical Tracking Solar Collection Skins in Architecture
NSF Convergence Accelerator Track M:建筑中高性能机械跟踪太阳能收集表皮的仿生表面设计
  • 批准号:
    2344424
  • 财政年份:
    2024
  • 资助金额:
    $ 52.6万
  • 项目类别:
    Standard Grant
Collaborative Research:CIF:Small:Fisher-Inspired Approach to Quickest Change Detection for Score-Based Models
合作研究:CIF:Small:Fisher 启发的基于评分模型的最快变化检测方法
  • 批准号:
    2334898
  • 财政年份:
    2024
  • 资助金额:
    $ 52.6万
  • 项目类别:
    Standard Grant
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了