Entropic risk minimization for nonparametric estimation of mixing distributions
Entropic risk minimization for nonparametric estimation of mixing distributions
复制标题
混合分布非参数估计的熵风险最小化
DOI：
10.1007/s10994-014-5467-7
复制
发表时间：
2015
期刊：
影响因子：
7.5
通讯作者：
Kazuho Watanabe and Shiro Ikeda
中科院分区：
文献类型：
--
作者：
石原直樹;久留美里織;竹内一郎;I. Takeuchi and M. Sugiyama;I. Takeuchi and M Sugiyama;Kazuho Watanabe and Shiro Ikeda
讨论混合模型中混合分布的一种非参数估计方法。该问题被形式化地表示为单参数目标泛函的最小化，在特殊情况下成为最大似然估计或核矢量量化。推广了非参数极大似然估计定理，证明了最优混合分布的存在性和离散性，并给出了计算最优混合分布的算法。结果表明，在适当选择参数的情况下，该方法比极大似然法更不容易发生过拟合。我们进一步讨论了统一估计框架和率失真问题之间的联系。
We discuss a nonparametric estimation method for the mixing distributions in mixture models. The problem is formalized as a minimization of a one-parameter objective functional, which becomes the maximum likelihood estimation or the kernel vector quantization in special cases. Generalizing the theorem for the nonparametric maximum likelihood estimation, we prove the existence and discreteness of the optimal mixing distribution and provide an algorithm to calculate it. It is demonstrated that with an appropriate choice of the parameter, the proposed method is less prone to overfitting than the maximum likelihood method. We further discuss the connection between the unifying estimation framework and the rate-distortion problem.