Debates: Does Information Theory Provide a New Paradigm for Earth Science? Sharper Predictions Using Occam's Digital Razor
Debates: Does Information Theory Provide a New Paradigm for Earth Science? Sharper Predictions Using Occam's Digital Razor
复制标题
辩论：信息论是否为地球科学提供了新范式？
DOI：
10.1029/2019wr026471
复制
发表时间：
2020
影响因子：
5.4
通讯作者：
B. Ruddell
中科院分区：
文献类型：
--
作者：
S. Weijs;B. Ruddell
奥卡姆剃刀是科学哲学的基本原则，指出在任何给定的模型预测性能水平下，最简单的假设（或模型）是首选的。一个经常被认为是爱因斯坦的现代重述解释说：“一切都应该尽可能简单，但不是更简单。”使用（算法）信息论的原理，模型描述性能和模型复杂性都可以用比特来量化。这种量化在模型复杂性（以比特为单位的模型程序长度）和模型性能（以比特为单位的信息损失，或描述原始观测所需的缺失信息）之间产生了帕累托式的权衡。模型的复杂性和性能可以被压缩为无损模型大小的一个单一度量，当最小化时，会导致最佳的模型复杂性与泛化和预测的损失权衡。我们的观点将简单的数据驱动模型和复杂的基于物理过程的模型放在一个连续体上，在这个意义上，两者都以压缩形式描述了观察数据中的模式，具有不同程度的通用性，模型复杂性和描述性能。基于信息论的压缩性能评估，以及对模型复杂性的公平和有意义的解释，将使我们能够在给定数据可用性的情况下，针对给定问题最好地比较和联合收割机物理知识和数据驱动建模的优势。
Occam's Razor is a bedrock principle of science philosophy, stating that the simplest hypothesis (or model) is preferred, at any given level of model predictive performance. A modern restatement often attributed to Einstein explains, “Everything should be made as simple as possible, but not simpler.” Using principles from (algorithmic) information theory, both model descriptive performance and model complexity can be quantified in bits. This quantification yields a Pareto‐style trade‐off between model complexity (length of the model program in bits) and model performance (information loss in bits, or the missing information, needed to describe the original observations). Model complexity and performance can be collapsed to one single measure of lossless model size, which, when minimized, leads to optimal model complexity versus loss trade‐off for generalization and prediction. Our view puts both simple data‐driven and complex physical‐process‐based models on a continuum, in the sense that both describe patterns in observed data in compressed form, with different degrees of generality, model complexity, and descriptive performance. Information theory‐based assessment of compression performance with fair and meaningful accounting for model complexity will enable us to best compare and combine the strengths of physics knowledge and data‐driven modeling for a given problem, given the availability of data.