Significance measure and result replicability in prediction modelling

结果可复制性被认为是科学研究方法论的基础。统计显著性不能评价结果的可重复性。作为衡量显著性的p值在统计分析和决策中占主导地位，但令人惊讶的是，它几乎没有说明可复制性。在一个典型实验的少量重复的模拟中，p值可以在很大的间隔内变化，这是很常见的。由此可见，p值是一个非常不可靠的度量。外部和内部可复制性分析都很重要。真正的外部复制研究很多时候是不切实际的，因此研究人员转向内部方法，如交叉验证、折刀和bootstrap。近年来，联结函数作为一种合适的随机变量间相关性度量而出现，为研究随机变量间的相关性提供了一种替代模型。

Result replicability is considered fundamental in scientific research methodology. Statistical significance does not evaluate result replicability. The p-value as a measure of significance dominates statistical analysis and decision making however it surprisingly tells little about replicability. It is not uncommon to observe that in one simulation of small number of repeats of a typical experiment, p-value can vary over a large interval. Thus illustrating that p-value is a very unreliable measure. Both external and internal replicability analyses are important. True external replication studies are many times impractical, and therefore researchers turn to internal methods like cross-validation, the jackknife and the bootstrap. Recently copula functions have emerged as an appropriate measure of dependence among random variables and provide an alternative model to the commonly practised correlation based models for studying the dependence.