Scalable Bayesian Meta-Learning through Generalized Implicit Gradients
Scalable Bayesian Meta-Learning through Generalized Implicit Gradients
复制标题
DOI：
10.1609/aaai.v37i9.26337
复制
发表时间：
2023-03
期刊：
影响因子：
--
通讯作者：
Yilang Zhang;Bingcong Li;Shi-Ji Gao;G. Giannakis
中科院分区：
文献类型：
--
作者：
Yilang Zhang;Bingcong Li;Shi-Ji Gao;G. Giannakis
元学习具有独特的有效性和迅速的能力，可以通过有限的数据来应对新兴任务。通过将其视为双层优化问题，可以揭示其广泛的适用性。但是，当内部级优化依赖于基于梯度的迭代时，最终的算法视图将面临可扩展性问题。隐性区分被认为是为了减轻这一挑战，但仅限于各向同性高斯之前，只有有利于确定性的元学习方法。这项工作明显通过使用隐式分化对概率贝叶斯元学习的互换的益处来显着降低可扩展性瓶颈。新型隐式贝叶斯元学习（IBAML）方法不仅扩大了可学习的先验的范围，而且还量化了相关的不确定性。此外，无论内部优化轨迹如何，最终的复杂性都可以很好地控制。建立了分析误差范围，以证明与显式相比，广义隐式梯度的精度和效率。还进行了广泛的数值测试，以验证所提出方法的性能。
Meta-learning owns unique effectiveness and swiftness in tackling emerging tasks with limited data. Its broad applicability is revealed by viewing it as a bi-level optimization problem. The resultant algorithmic viewpoint however, faces scalability issues when the inner-level optimization relies on gradient-based iterations. Implicit differentiation has been considered to alleviate this challenge, but it is restricted to an isotropic Gaussian prior, and only favors deterministic meta-learning approaches. This work markedly mitigates the scalability bottleneck by cross-fertilizing the benefits of implicit differentiation to probabilistic Bayesian meta-learning. The novel implicit Bayesian meta-learning (iBaML) method not only broadens the scope of learnable priors, but also quantifies the associated uncertainty. Furthermore, the ultimate complexity is well controlled regardless of the inner-level optimization trajectory. Analytical error bounds are established to demonstrate the precision and efficiency of the generalized implicit gradient over the explicit one. Extensive numerical tests are also carried out to empirically validate the performance of the proposed method.