Meta Learning in Decentralized Neural Networks: Towards More General AI
Meta Learning in Decentralized Neural Networks: Towards More General AI
复制标题
DOI：
10.48550/arxiv.2302.01020
复制
发表时间：
2023-02
期刊：
影响因子：
--
通讯作者：
Yuwei Sun
中科院分区：
文献类型：
--
作者：
Yuwei Sun
元学习通常是指从其他学习算法中学习的学习算法。神经网络预测中的不确定性问题表明，世界只能部分预测，并且学习过的神经网络无法推广到其不断变化的周围环境。因此，问题是预测模型如何同时表示多个预测。我们的目标是在分散式神经网络（Decentralized NN）的内容中提供对学习的基本理解，我们相信这是构建自主智能机器的最重要问题和先决条件之一。为此，我们将展示几个证据，用于在分散式NN中使用Meta学习来解决上述问题。特别是，我们将提出三种不同的方法来构建这样一个分散的学习系统：（1）从许多副本神经网络中学习，（2）为不同的功能构建神经网络的层次结构，以及（3）利用不同的模态专家来学习跨模态表示。
Meta-learning usually refers to a learning algorithm that learns from other learning algorithms. The problem of uncertainty in the predictions of neural networks shows that the world is only partially predictable and a learned neural network cannot generalize to its ever-changing surrounding environments. Therefore, the question is how a predictive model can represent multiple predictions simultaneously. We aim to provide a fundamental understanding of learning to learn in the contents of Decentralized Neural Networks (Decentralized NNs) and we believe this is one of the most important questions and prerequisites to building an autonomous intelligence machine. To this end, we shall demonstrate several pieces of evidence for tackling the problems above with Meta Learning in Decentralized NNs. In particular, we will present three different approaches to building such a decentralized learning system: (1) learning from many replica neural networks, (2) building the hierarchy of neural networks for different functions, and (3) leveraging different modality experts to learn cross-modal representations.