Empirical Bayes via ERM and Rademacher complexities: the Poisson model
Empirical Bayes via ERM and Rademacher complexities: the Poisson model
复制标题
DOI：
--
复制
发表时间：
2023-07
影响因子：
3.2
通讯作者：
Soham Jana;Yury Polyanskiy;Anzo Teh;Yihong Wu
中科院分区：
文献类型：
--
作者：
Soham Jana;Yury Polyanskiy;Anzo Teh;Yihong Wu
本文研究了（多元）Poisson均值的经验Bayes估计问题。现有的解决方案，已被证明是理论上最佳的最小化遗憾（超过贝叶斯预言，知道先验的过度风险）有几个缺点。例如，经典的Robbins估计不保持贝叶斯估计的单调性，并且在中等样本量下表现不佳。基于最小距离和非参数最大似然（NPMLE）方法的估计纠正了这些问题，但计算成本高，复杂性随维数呈指数增长。扩展了Barbehenn和Zhao（2022）的方法，在这项工作中，我们基于经验风险最小化（ERM）构建了单调估计量，这些估计量保留了类似的理论保证，并且可以更有效地计算。将偏移Rademacher复杂性Liang et al.（2015）的思想适应于经验贝叶斯中的非标准损失和函数类，我们证明了形状约束的ERM估计量在一维的常数因子内和多维的对数因子内获得了极大极小遗憾。
We consider the problem of empirical Bayes estimation for (multivariate) Poisson means. Existing solutions that have been shown theoretically optimal for minimizing the regret (excess risk over the Bayesian oracle that knows the prior) have several shortcomings. For example, the classical Robbins estimator does not retain the monotonicity property of the Bayes estimator and performs poorly under moderate sample size. Estimators based on the minimum distance and non-parametric maximum likelihood (NPMLE) methods correct these issues, but are computationally expensive with complexity growing exponentially with dimension. Extending the approach of Barbehenn and Zhao (2022), in this work we construct monotone estimators based on empirical risk minimization (ERM) that retain similar theoretical guarantees and can be computed much more efficiently. Adapting the idea of offset Rademacher complexity Liang et al. (2015) to the non-standard loss and function class in empirical Bayes, we show that the shape-constrained ERM estimator attains the minimax regret within constant factors in one dimension and within logarithmic factors in multiple dimensions.