Learning Functional Transduction
Learning Functional Transduction
复制标题
DOI：
10.48550/arxiv.2302.00328
复制
发表时间：
2023-02
期刊：
影响因子：
--
通讯作者：
Mathieu Chalvidal;Thomas Serre;Rufin VanRullen
中科院分区：
文献类型：
--
作者：
Mathieu Chalvidal;Thomas Serre;Rufin VanRullen
机器学习的研究已经两极分化为两种用于回归任务的通用方法： 传导方法直接从可用数据构建估计，但通常问题不具体。归纳方法可以更加具体，但通常需要计算密集型解决方案搜索。在这项工作中，我们提出了一种混合方法，并表明转导回归原理可以通过梯度下降进行元学习，以利用向量值再生核巴纳空间（RKBS）的理论形成有效的上下文神经逼近器。我们将这种方法应用于在有限和无限维空间（函数值运算符）上定义的函数空间，并表明一旦经过训练，传感器几乎可以在给定几对输入和输出示例的情况下立即捕获无限的函数关系，并返回新的图像估计。我们展示了元学习转导方法的优势，可以用很少的数据来模拟受不同外部因素影响的复杂物理系统，而成本仅为偏微分方程和气候建模应用的通常深度学习训练计算成本的一小部分。
Research in machine learning has polarized into two general approaches for regression tasks: Transductive methods construct estimates directly from available data but are usually problem unspecific. Inductive methods can be much more specific but generally require compute-intensive solution searches. In this work, we propose a hybrid approach and show that transductive regression principles can be meta-learned through gradient descent to form efficient in-context neural approximators by leveraging the theory of vector-valued Reproducing Kernel Banach Spaces (RKBS). We apply this approach to function spaces defined over finite and infinite-dimensional spaces (function-valued operators) and show that once trained, the Transducer can almost instantaneously capture an infinity of functional relationships given a few pairs of input and output examples and return new image estimates. We demonstrate the benefit of our meta-learned transductive approach to model complex physical systems influenced by varying external factors with little data at a fraction of the usual deep learning training computational cost for partial differential equations and climate modeling applications.