Optimal rates of entropy estimation over Lipschitz balls
Optimal rates of entropy estimation over Lipschitz balls
复制标题
DOI：
10.1214/19-aos1927
复制
发表时间：
2017-11
期刊：
影响因子：
--
通讯作者：
Yanjun Han;Jiantao Jiao;T. Weissman;Yihong Wu
中科院分区：
文献类型：
--
作者：
Yanjun Han;Jiantao Jiao;T. Weissman;Yihong Wu
我们考虑了在Lipschitz球上密度的熵的最小值估计的问题。丢弃一个通常的假设，即密度远离零，我们获得了最小值$（n \ ln n）^{ - \ frac {s} {s + d}}}}} + n^{ - 1/2} $ for $ 0 <s \ leq 2 $在任意尺寸$ d $中，其中$ s $是平滑度参数，$ n $是独立样本的数量。使用两阶段近似技术，该技术首先通过其核平滑版本近似密度，然后通过多项式近似于非平滑函数，我们构造了达到最小收敛速率的熵估计器，通过匹配下限显示了最佳的收敛速率。分析偏见的关键步骤之一依赖于强大的小木最大不平等现象的新应用，这也导致了可能具有独立关注的Fisher信息的新不平等。
We consider the problem of minimax estimation of the entropy of a density over Lipschitz balls. Dropping the usual assumption that the density is bounded away from zero, we obtain the minimax rates $(n\ln n)^{-\frac{s}{s+d}} + n^{-1/2}$ for $0<s\leq 2$ in arbitrary dimension $d$, where $s$ is the smoothness parameter and $n$ is the number of independent samples. Using a two-stage approximation technique, which first approximate the density by its kernel-smoothed version, and then approximate the non-smooth functional by polynomials, we construct entropy estimators that attain the minimax rate of convergence, shown optimal by matching lower bounds. One of the key steps in analyzing the bias relies on a novel application of the Hardy-Littlewood maximal inequality, which also leads to a new inequality on Fisher information that might be of independent interest.