CDS&E/Collaborative Research: A New Framework for Computational Model Validation

模拟在帮助决策者应对复杂的社会问题方面发挥着关键作用，这些问题包括提高基础设施抵御自然灾害能力的战略、了解金融市场或管理空中交通。然而，为了有效，个人必须选择合适的模型并了解其局限性。该项目进行基础研究，为模型验证创建一个计算效率高的范例，旨在指导模型准确性和有效性的决策。由此产生的概率框架将允许用户探索许多模型家族，并选择最有用的模型。先进的计算算法和架构将使该框架能够利用来自设备、平台、传感器、系统和用户的大量数据，同时评估多个模型。通过开发模型证伪和贝叶斯模型选择的综合框架，项目研究者为研究人员和决策者提供了一个强大的工具，可以有效地使用模型来理解和应对新出现的社会挑战和机遇。将贝叶斯模型选择与两个模型证伪阶段相结合，形成了一种新的模型验证概率框架：初始证伪消除不适合测量数据的模型，而最终证伪选择具有最准确模拟的模型类。模型证伪的新方法——错误发现率测试和模型似然阈值——提供了一个内聚框架，可以容纳多种类型的动态响应数据。该方法通过测试平台建模问题进行了优化和校准，例如NASA的湍流模型验证基准和完全仪器化的基地隔离建筑建模。这项工作将改变模型验证的效率、准确性和广泛适用性，使其能够用于以前难以处理的科学和工程应用。研究人员将在有前途的领域探索建模合作，包括湍流、材料行为和生化反应。该项目将通过教育和外联活动提高对模型验证的认识。研究成果将在期刊出版物中传播，并在国家和国际会议上发表演讲。将指导两名研究生，并将研究概念和成果纳入两门研究生不确定性量化课程和一门本科生工程风险分析课程。研究人员还将把研究结果纳入他们正在进行的K-12外展工作中，包括针对初中/高中STEM丰富课程的模型构建模块。

Simulations play a key role in helping decision makers respond to complex societal problems, ranging from strategies for enhancing infrastructure resilience to natural hazards, to understanding financial markets, or managing air traffic. However, to be effective, individuals must select appropriate models and understand their limitations. This project conducts fundamental research to create a computationally efficient paradigm for model validation designed to guide decisions on model accuracy and validity. The resulting probabilistic framework will allow users to explore many families of models and to choose the ones that are the most useful. Advanced computational algorithms and architectures will enable the framework to leverage vast amounts of data from devices, platforms, sensors and systems and users to simultaneously evaluate multiple models. By developing an integrated framework of model falsification and Bayesian model selection, the project investigators give researchers and decision makers a powerful tool for effectively using models to understand and respond to emerging societal challenges and opportunities. Integrating Bayesian model selection with two model falsification phases forms a novel probabilistic framework for model validation: initial falsification eliminates unsuitable models that do not fit measured data, while final falsification selects the model classes with the most accurate simulations. New approaches to model falsification -- false discovery rate testing and model likelihood thresholds -- provide a cohesive framework that accommodates many types of dynamic response data. The approach is optimized and calibrated with testbed modeling problems such as NASA's benchmark for turbulence model validation and modeling a fully instrumented base isolated building. This work will transform the efficiency, accuracy and widespread applicability of model validation, allowing it to be used for previously intractable science and engineering applications. The researchers will explore modeling collaborations in promising areas including turbulence, material behavior, and biochemical reactions. The project will enhance awareness of model validation through education and outreach activities. Research results will be disseminated in journal publications and presentations at national and international conferences. Two graduate students will be mentored and research concepts and results will be incorporated into two graduate uncertainty quantification courses and an undergraduate engineering risk analysis class. The researchers will also incorporate research findings into their ongoing K-12 outreach efforts, including a model-building module targeting middle/high school STEM enrichment programs.