Effectiveness of Kullback-Leibler Information As A Measure of Dependence

Kullback-Leibler 信息作为依赖性衡量标准的有效性

基本信息

批准号：
12480063
负责人：
SHIBATA Ritei
金额：
$ 6.66万
依托单位：
Keio University
依托单位国家：
日本
项目类别：
Grant-in-Aid for Scientific Research (B)
财政年份：
2000
资助国家：
日本
起止时间：
2000 至 2002
项目状态：
已结题

项目摘要

The aim of this project is to investigate effectiveness of Kullback-Leibler information. In this project, various aspects of this information measure have been investigated.We could show the effectiveness of Kullback-Leibler information as a criterion of model selection. It is clarified that Bootstrap type estimate of Kullback-Leibler information is quite powerful, particularly in case of discrete distributions like Binomial or Multinomial.To ensure practical usefulness of model selection technique based on Kullback-Leibler information, we performed various type of real data analysis, In due course of analysis of interest rate time series, we found that neural network should be included in a family of statistical models to be selected. We then extended ordinary neural network to stochastic neural network and developed an efficient training algorithm. We also gave a mathematical proof of the convergence. The stochastic neural network is quite powerful, for example, it gives us the best one day ahead prediction of fall or rise of TOPIX with around 60% accuracy.We also analyzed satellite radar received signals and instantaneous foreign exchange prices to investigate effectiveness of Kullback-Leibler information as a criterion for the processing. As a result, we found ten times precise data processing algorithm for the former and constructed a clustered Poisson marked process for the latter.To investigate information flows on graphical model, we concentrated our attention into conditional independence which is a key idea in graphical modeling. As a result, we found that conditional independence is too strong condition to be realized unless in case of normal distribution or its monotone transformed distribution. However, we found that Kullback-Leibler information is a promising alternative measure in place of conditional independence.

该项目的目的是研究Kullback-Leibler信息的有效性。在该项目中，已经研究了此信息度量的各个方面。我们可以将Kullback-Leibler信息作为模型选择标准的有效性。可以澄清说，kullback-leibler信息的自举类型估计非常强大，尤其是在诸如二项式或多项式的离散分布的情况下。为了确保基于Kullback-Leibler信息的模型选择技术的实际实用性，我们在适当的数据分析中，我们发现了对Never Inter网络的适当分析，我们在统计中选择了一个统计的模型。然后，我们将普通的神经网络扩展到随机神经网络，并开发了有效的训练算法。我们还给出了融合的数学证明。随机神经网络非常强大，例如，它为我们提供了最佳的一天预测Topix的前一天，其准确性约为60％。我们还分析了卫星雷达收到的信号和瞬时外汇价格，以研究Kullback-Leibler信息作为处理标准的Kullback-Leibler信息的有效性。结果，我们发现了前者的十倍精确的数据处理算法，并构建了一个群集的泊松标记过程的过程。为了研究图形模型的信息流，我们将注意力集中在有条件的独立性上，这是图形建模的关键思想。结果，我们发现有条件的独立性太强，除非出现正常分布或其单调转换分布，否则有条件的独立性。但是，我们发现Kullback-Leibler信息是一个有前途的替代措施，而不是有条件的独立性。