An Optimization Approach of Deriving Bounds between Entropy and Error from Joint Distribution: Case Study for Binary Classifications
An Optimization Approach of Deriving Bounds between Entropy and Error from Joint Distribution: Case Study for Binary Classifications
复制标题
从联合分布导出熵和误差之间界限的优化方法：二元分类的案例研究
DOI：
10.3390/e18020059
复制
发表时间：
2016-02
期刊：
影响因子：
2.7
通讯作者：
Xing Hong-Jie
中科院分区：
文献类型：
--
作者：
Hu Bao-Gang;Xing Hong-Jie
在这项工作中，我们提出了一种新的方法，通过优化手段从联合分布中推导出熵和误差之间的界限。具体的案例研究给出了二进制分类。两种基本类型的分类错误的调查，即贝叶斯和非贝叶斯错误。考虑非贝叶斯错误是由于大多数分类器导致非贝叶斯解决方案的事实。对于这两种类型的错误，我们推导出每个边界和错误组件之间的封闭形式的关系。在此基础上实现了“错误概率与条件熵”图中的Fano下界，并通过引入非贝叶斯错误和两种沿着变量独立性质的情况，扩大了对Fano下界的解释。贝叶斯误差的一个新的上限推导出相对于最小先验概率，这通常是严格的Kovalevskij的上限。
In this work, we propose a new approach of deriving the bounds between entropy and error from a joint distribution through an optimization means. The specific case study is given on binary classifications. Two basic types of classification errors are investigated, namely, the Bayesian and non-Bayesian errors. The consideration of non-Bayesian errors is due to the facts that most classifiers result in non-Bayesian solutions. For both types of errors, we derive the closed-form relations between each bound and error components. When Fano’s lower bound in a diagram of “Error Probability vs. Conditional Entropy” is realized based on the approach, its interpretations are enlarged by including non-Bayesian errors and the two situations along with independent properties of the variables. A new upper bound for the Bayesian error is derived with respect to the minimum prior probability, which is generally tighter than Kovalevskij’s upper bound.
登录
查看更多内容
影响因子：
2.5
作者：
Harremoës, P;Topsoe, F
通讯作者：
Topsoe, F
影响因子：
3.9
作者：
Eriksson, T;Kim, S;Lee, C
通讯作者：
Lee, C
DOI：
10.1109/dsp.2009.4785948
发表时间：
2009-02
期刊：
2009 IEEE 13th Digital Signal Processing Workshop and 5th IEEE Signal Processing Education Workshop
影响因子：
--
作者：
John W. Fisher III;Michael R. Siracusa;Kinh H. Tieu
通讯作者：
John W. Fisher III;Michael R. Siracusa;Kinh H. Tieu
DOI：
10.1017/cbo9780511801389.013
发表时间：
2000-03
期刊：
--
影响因子：
--
作者：
N. Cristianini;J. Shawe-Taylor
通讯作者：
N. Cristianini;J. Shawe-Taylor
影响因子：
--
作者：
R. Fano
通讯作者：
R. Fano