Effectiveness of Kullback-Leibler Information As A Measure of Dependence

点击翻译按钮获取中文摘要

The aim of this project is to investigate effectiveness of Kullback-Leibler information. In this project, various aspects of this information measure have been investigated.We could show the effectiveness of Kullback-Leibler information as a criterion of model selection. It is clarified that Bootstrap type estimate of Kullback-Leibler information is quite powerful, particularly in case of discrete distributions like Binomial or Multinomial.To ensure practical usefulness of model selection technique based on Kullback-Leibler information, we performed various type of real data analysis, In due course of analysis of interest rate time series, we found that neural network should be included in a family of statistical models to be selected. We then extended ordinary neural network to stochastic neural network and developed an efficient training algorithm. We also gave a mathematical proof of the convergence. The stochastic neural network is quite powerful, for example, it gives us the best one day ahead prediction of fall or rise of TOPIX with around 60% accuracy.We also analyzed satellite radar received signals and instantaneous foreign exchange prices to investigate effectiveness of Kullback-Leibler information as a criterion for the processing. As a result, we found ten times precise data processing algorithm for the former and constructed a clustered Poisson marked process for the latter.To investigate information flows on graphical model, we concentrated our attention into conditional independence which is a key idea in graphical modeling. As a result, we found that conditional independence is too strong condition to be realized unless in case of normal distribution or its monotone transformed distribution. However, we found that Kullback-Leibler information is a promising alternative measure in place of conditional independence.