Effectiveness of Kullback-Leibler Information As A Measure of Dependence

本项目的目的是调查Kullback-Leibler信息的有效性。在这个项目中，我们对这种信息度量的各个方面进行了研究，我们可以证明Kullback-Leibler信息作为模型选择的标准是有效的。为了保证基于Kullback-Leibler信息的模型选择技术的实用性，我们进行了各种类型的实际数据分析，在对利率时间序列进行分析的过程中，我们发现神经网络应该包括在一类统计模型中进行选择。然后将普通神经网络扩展到随机神经网络，并提出了一种有效的训练算法。我们还给出了收敛的一个数学证明。随机神经网络是非常强大的，例如，它给出了对东证指数下跌或上涨的最好提前一天的预测，准确率约为60%。我们还分析了卫星雷达接收的信号和瞬时外汇价格，以考察Kullback-Leibler信息作为处理标准的有效性。为了研究图形化模型上的信息流，我们将注意力集中在图形化建模的关键思想--条件独立性上。结果，我们发现条件独立性的条件太强，除非是正态分布或其单调变换分布。然而，我们发现Kullback-Leibler信息是一种很有前途的替代条件独立性的衡量标准。

The aim of this project is to investigate effectiveness of Kullback-Leibler information. In this project, various aspects of this information measure have been investigated.We could show the effectiveness of Kullback-Leibler information as a criterion of model selection. It is clarified that Bootstrap type estimate of Kullback-Leibler information is quite powerful, particularly in case of discrete distributions like Binomial or Multinomial.To ensure practical usefulness of model selection technique based on Kullback-Leibler information, we performed various type of real data analysis, In due course of analysis of interest rate time series, we found that neural network should be included in a family of statistical models to be selected. We then extended ordinary neural network to stochastic neural network and developed an efficient training algorithm. We also gave a mathematical proof of the convergence. The stochastic neural network is quite powerful, for example, it gives us the best one day ahead prediction of fall or rise of TOPIX with around 60% accuracy.We also analyzed satellite radar received signals and instantaneous foreign exchange prices to investigate effectiveness of Kullback-Leibler information as a criterion for the processing. As a result, we found ten times precise data processing algorithm for the former and constructed a clustered Poisson marked process for the latter.To investigate information flows on graphical model, we concentrated our attention into conditional independence which is a key idea in graphical modeling. As a result, we found that conditional independence is too strong condition to be realized unless in case of normal distribution or its monotone transformed distribution. However, we found that Kullback-Leibler information is a promising alternative measure in place of conditional independence.