Optimal Rates for Regularized Conditional Mean Embedding Learning
Optimal Rates for Regularized Conditional Mean Embedding Learning
复制标题
正则化条件均值嵌入学习的最佳速率
DOI：
10.48550/arxiv.2208.01711
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
A. Gretton
中科院分区：
文献类型：
--
作者：
Zhu Li;D. Meunier;Mattes Mollenhauer;A. Gretton
我们讨论了条件平均嵌入（CME）的核脊回归估计的一致性，CME是将给定$X$的条件分布$Y$嵌入到目标再现核希尔伯特空间$\mathcal{H}_Y$中。CME允许我们取目标RKHS函数的条件期望，并已用于非参数因果推理和贝叶斯推理。我们解决了错误的设置，其中目标CME在Hilbert-Schmidt算子空间中，从$\mathcal{H}_X$和$L_2$之间的输入插值空间作用到$\mathcal{H}_Y$。这个算子空间被证明是同构于一个新定义的向量值插值空间。利用这一同构性，我们得到了错误设定下经验CME估计量的一种新的自适应统计学习率。我们的分析表明，我们的速率匹配最优的$O(\log n / n)$速率，而不假设$\mathcal{H}_Y$是有限维的。我们进一步建立了学习率的下界，表明得到的上界是最优的。
We address the consistency of a kernel ridge regression estimate of the conditional mean embedding (CME), which is an embedding of the conditional distribution of $Y$ given $X$ into a target reproducing kernel Hilbert space $\mathcal{H}_Y$. The CME allows us to take conditional expectations of target RKHS functions, and has been employed in nonparametric causal and Bayesian inference. We address the misspecified setting, where the target CME is in the space of Hilbert-Schmidt operators acting from an input interpolation space between $\mathcal{H}_X$ and $L_2$, to $\mathcal{H}_Y$. This space of operators is shown to be isomorphic to a newly defined vector-valued interpolation space. Using this isomorphism, we derive a novel and adaptive statistical learning rate for the empirical CME estimator under the misspecified setting. Our analysis reveals that our rates match the optimal $O(\log n / n)$ rates without assuming $\mathcal{H}_Y$ to be finite dimensional. We further establish a lower bound on the learning rate, which shows that the obtained upper bound is optimal.