CDS&E/Collaborative Research: A New Framework for Computational Model Validation

模拟在帮助决策者应对复杂的社会问题方面发挥着关键作用，从增强基础设施抵御自然灾害的能力到了解金融市场或管理空中交通。 然而，为了有效，个人必须选择适当的模式并了解其局限性。 该项目进行基础研究，为模型验证创建一个计算效率高的范例，旨在指导模型准确性和有效性的决策。 由此产生的概率框架将允许用户探索许多系列的模型，并选择最有用的模型。 先进的计算算法和架构将使该框架能够利用来自设备、平台、传感器和系统以及用户的大量数据，同时评估多个模型。 通过开发模型证伪和贝叶斯模型选择的综合框架，项目研究人员为研究人员和决策者提供了一个强大的工具，可以有效地使用模型来理解和应对新兴的社会挑战和机遇。将贝叶斯模型选择与两个模型证伪阶段相结合，形成了一个新的模型验证概率框架：初始证伪消除了不适合测量数据的不合适的模型，而最终证伪选择了具有最准确模拟的模型类。 模型证伪的新方法-错误发现率测试和模型似然阈值-提供了一个内聚框架，可容纳多种类型的动态响应数据。 该方法进行了优化和校准的试验台建模问题，如美国宇航局的基准湍流模型验证和建模一个完全仪表基地隔离的建筑物。 这项工作将改变模型验证的效率，准确性和广泛的适用性，使其能够用于以前棘手的科学和工程应用。 研究人员将探索在有前途的领域，包括湍流，材料行为和生化反应建模合作。 该项目将通过教育和外联活动提高对模型验证的认识。研究结果将在期刊出版物中传播，并在国家和国际会议上发表。两名研究生将接受指导，研究概念和结果将纳入两个研究生不确定性量化课程和本科工程风险分析课程。 研究人员还将把研究结果纳入他们正在进行的K-12外展工作中，包括一个针对初中/高中STEM富集计划的模型构建模块。

Simulations play a key role in helping decision makers respond to complex societal problems, ranging from strategies for enhancing infrastructure resilience to natural hazards, to understanding financial markets, or managing air traffic. However, to be effective, individuals must select appropriate models and understand their limitations. This project conducts fundamental research to create a computationally efficient paradigm for model validation designed to guide decisions on model accuracy and validity. The resulting probabilistic framework will allow users to explore many families of models and to choose the ones that are the most useful. Advanced computational algorithms and architectures will enable the framework to leverage vast amounts of data from devices, platforms, sensors and systems and users to simultaneously evaluate multiple models. By developing an integrated framework of model falsification and Bayesian model selection, the project investigators give researchers and decision makers a powerful tool for effectively using models to understand and respond to emerging societal challenges and opportunities.Integrating Bayesian model selection with two model falsification phases forms a novel probabilistic framework for model validation: initial falsification eliminates unsuitable models that do not fit measured data, while final falsification selects the model classes with the most accurate simulations. New approaches to model falsification -- false discovery rate testing and model likelihood thresholds -- provide a cohesive framework that accommodates many types of dynamic response data. The approach is optimized and calibrated with testbed modeling problems such as NASA's benchmark for turbulence model validation and modeling a fully instrumented base isolated building. This work will transform the efficiency, accuracy and widespread applicability of model validation, allowing it to be used for previously intractable science and engineering applications. The researchers will explore modeling collaborations in promising areas including turbulence, material behavior, and biochemical reactions. The project will enhance awareness of model validation through education and outreach activities. Research results will be disseminated in journal publications and presentations at national and international conferences. Two graduate students will be mentored and research concepts and results will be incorporated into two graduate uncertainty quantification courses and an undergraduate engineering risk analysis class. The researchers will also incorporate research findings into their ongoing K-12 outreach efforts, including a model-building module targeting middle/high school STEM enrichment programs.