Significance measure and result replicability in prediction modelling

点击翻译按钮获取中文摘要

Result replicability is considered fundamental in scientific research methodology. Statistical significance does not evaluate result replicability. The p-value as a measure of significance dominates statistical analysis and decision making however it surprisingly tells little about replicability. It is not uncommon to observe that in one simulation of small number of repeats of a typical experiment, p-value can vary over a large interval. Thus illustrating that p-value is a very unreliable measure. Both external and internal replicability analyses are important. True external replication studies are many times impractical, and therefore researchers turn to internal methods like cross-validation, the jackknife and the bootstrap. Recently copula functions have emerged as an appropriate measure of dependence among random variables and provide an alternative model to the commonly practised correlation based models for studying the dependence.