Information geometry for multiparameter models: new perspectives on the origin of simplicity
Information geometry for multiparameter models: new perspectives on the origin of simplicity
复制标题
DOI：
10.1088/1361-6633/aca6f8
复制
发表时间：
2021-11
影响因子：
18.1
通讯作者：
Katherine N. Quinn;Michael C. Abbott;M. Transtrum;B. Machta;J. Sethna
中科院分区：
文献类型：
--
作者：
Katherine N. Quinn;Michael C. Abbott;M. Transtrum;B. Machta;J. Sethna
物理学、生物学、经济学和工程学中的复杂模型往往是草率的，这意味着模型参数不能很好地由模型对集体行为的预测来确定。许多参数组合可以在几十年内变化，而预测结果没有显著变化。这篇综述使用信息几何来探讨马虎及其与涌现理论的深层关系。引入预测的模型流形，其坐标为模型参数。它的超丝带结构解释了为什么只有几个参数组合对行为有影响。我们回顾了最近将超带宽度的层次结构与近似理论以及控制变量变化下模型预测的平滑性联系起来的严谨结果。我们讨论了最近的测地线方法，以在模型流形的附近边界上找到更简单的模型-具有更少参数的涌现理论，可以很好地解释行为。我们讨论了一个贝叶斯先验，它优化了模型参数和实验数据之间的相互信息，自然地有利于出现边界理论上的点，从而更简单的模型。我们引入了一个“投影最大似然”先验，它有效地逼近了这个最优先验，并与传统杰弗里斯先验的不良行为进行了对比。讨论了统计力学中的重正化群粗粒化引入模型流形流的方式，并将模型流形上的僵硬和粗糙方向与重正化群的相关和不相关特征方向连接起来。最后，我们讨论了最近开发的“密集”嵌入方法，允许人们将任意概率模型的预测可视化为等距嵌入的低维投影，并通过生成Ising模型的模型流形来说明我们的方法。
Complex models in physics, biology, economics, and engineering are often sloppy, meaning that the model parameters are not well determined by the model predictions for collective behavior. Many parameter combinations can vary over decades without significant changes in the predictions. This review uses information geometry to explore sloppiness and its deep relation to emergent theories. We introduce the model manifold of predictions, whose coordinates are the model parameters. Its hyperribbon structure explains why only a few parameter combinations matter for the behavior. We review recent rigorous results that connect the hierarchy of hyperribbon widths to approximation theory, and to the smoothness of model predictions under changes of the control variables. We discuss recent geodesic methods to find simpler models on nearby boundaries of the model manifold—emergent theories with fewer parameters that explain the behavior equally well. We discuss a Bayesian prior which optimizes the mutual information between model parameters and experimental data, naturally favoring points on the emergent boundary theories and thus simpler models. We introduce a ‘projected maximum likelihood’ prior that efficiently approximates this optimal prior, and contrast both to the poor behavior of the traditional Jeffreys prior. We discuss the way the renormalization group coarse-graining in statistical mechanics introduces a flow of the model manifold, and connect stiff and sloppy directions along the model manifold with relevant and irrelevant eigendirections of the renormalization group. Finally, we discuss recently developed ‘intensive’ embedding methods, allowing one to visualize the predictions of arbitrary probabilistic models as low-dimensional projections of an isometric embedding, and illustrate our method by generating the model manifold of the Ising model.