Framework for Statistical Evaluation of Complex Computer Models

复杂计算机模型统计评估框架

• 批准号: 0073952
• 负责人: Jerome Sacks
• 金额: $ 90万
• 依托单位: National Institute of Statistical Sciences
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2000
• 资助国家: 美国
• 起止时间: 2000-08-01 至 2004-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0073952&HistoricalAwards=false
• 关键词: Framework; Statistical; Evaluation; Complex; Computer

Though inherently a statistical issue, model evaluation lacks a unifying statistical framework; this project supplies such. The foundation of the framework is the use of Bayesian techniques to quantify the degree to which a model captures an underlying reality; develop theory and methods that allow dual use of data in both estimation of model inputs and evaluation of outputs; select evaluation functions, by which a model and reality are compared and through which flaws (causes of "invalidity") are found; and to design the collection of field and computer simulation data. The building of the framework relies on specific formulations of problems, motivated by testbed examples such as subsurface fluid flow models - where the computer model is deterministic but uncertainties are present in the model inputs and specifications - and traffic simulators - where the model is intrinsically stochastic, in addition to having uncertain inputs. Computer models are everywhere and evaluating their fidelity to reality is central to assessing their effectiveness in understanding real phenomena (such as flow of pollutants through soil and into groundwater) and predicting results of innovative technologies (such as new signal timing strategies to relieve traffic congestion in urban networks). This project develops a statistical structure and basis for such evaluations applicable across the scientific and technological landscape. In the process, the project creates the ingredients for a virtual laboratory to disseminate results, broaden involvement of other researchers (and users), and establish a unique educational and training environment.

虽然本质上是一个统计问题，模型评估缺乏一个统一的统计框架，这个项目提供了这样的。该框架的基础是使用贝叶斯技术来量化模型捕捉潜在现实的程度;发展理论和方法，允许在估计模型输入和评价输出时双重使用数据;选择评价函数，通过该函数比较模型和现实，并通过该函数确定缺陷。（“无效”的原因）;并设计现场和计算机模拟数据的收集。该框架的构建依赖于问题的特定公式，由测试台示例激发，例如地下流体流动模型-其中计算机模型是确定性的，但模型输入和规范中存在不确定性-和交通模拟器-其中模型本质上是随机的，除了具有不确定的输入。 计算机模型无处不在，评估其对现实的保真度对于评估其在理解真实的现象（例如污染物通过土壤流入地下水）和预测创新技术（例如缓解城市网络交通拥堵的新信号配时策略）结果方面的有效性至关重要。该项目为适用于整个科学和技术领域的此类评价建立了统计结构和基础。 在此过程中，该项目为虚拟实验室创造了传播成果的要素，扩大了其他研究人员（和用户）的参与，并建立了独特的教育和培训环境。