Framework for Statistical Evaluation of Complex Computer Models

虽然模型评估本身是一个统计问题，但缺乏统一的统计框架；本项目提供了这种框架。该框架的基础是使用贝叶斯技术来量化模型对基本现实的捕捉程度；开发理论和方法，以便在估计模型投入和评价产出时能够同时使用数据；选择评估函数，通过该函数比较模型和现实，并通过该函数发现缺陷(“无效”的原因)；以及设计实地和计算机模拟数据的收集。框架的建立依赖于具体的问题表述，受地下流体流动模型和交通模拟器等试验台实例的推动-其中计算机模型是确定性的，但模型输入和规范中存在不确定性-其中模型本质上是随机的，除了具有不确定的输入。计算机模型无处不在，评估它们与现实的保真度对于评估它们在理解真实现象(如污染物通过土壤流入地下水)和预测创新技术的结果(如缓解城市网络交通拥堵的新信号配时策略)方面的有效性至关重要。该项目制定了适用于整个科学和技术领域的此类评价的统计结构和基础。在这一过程中，该项目为虚拟实验室创造了传播成果的要素，扩大了其他研究人员(和用户)的参与，并建立了一个独特的教育和培训环境。

Though inherently a statistical issue, model evaluation lacks a unifying statistical framework; this project supplies such. The foundation of the framework is the use of Bayesian techniques to quantify the degree to which a model captures an underlying reality; develop theory and methods that allow dual use of data in both estimation of model inputs and evaluation of outputs; select evaluation functions, by which a model and reality are compared and through which flaws (causes of "invalidity") are found; and to design the collection of field and computer simulation data. The building of the framework relies on specific formulations of problems, motivated by testbed examples such as subsurface fluid flow models - where the computer model is deterministic but uncertainties are present in the model inputs and specifications - and traffic simulators - where the model is intrinsically stochastic, in addition to having uncertain inputs. Computer models are everywhere and evaluating their fidelity to reality is central to assessing their effectiveness in understanding real phenomena (such as flow of pollutants through soil and into groundwater) and predicting results of innovative technologies (such as new signal timing strategies to relieve traffic congestion in urban networks). This project develops a statistical structure and basis for such evaluations applicable across the scientific and technological landscape. In the process, the project creates the ingredients for a virtual laboratory to disseminate results, broaden involvement of other researchers (and users), and establish a unique educational and training environment.