CAREER: Belief Formation in Verification Strategies and the Evaluation of Verification Evidence

在系统工程中，验证是建立对系统正常运行的信心的过程。虽然这是工程中的一项关键活动，但工程师如何利用客观数据来告知在验证过程中对系统的固有主观判断的基本原理还没有得到很好的理解。该教师早期职业发展计划（CAREER）奖支持对这一过程的基础研究和教育工作，旨在提高对验证活动的理解和技能。研究的重点是对信息结构进行建模，通过这些信息结构，工程师可以对系统的性能建立信心。这一知识可导致改进核查战略和更严格的方法来评估核查数据。因此，研究活动可以为系统工程的实践带来潜在的变革性范式。改进的核查战略使社会受益，因为它能够以更强有力的核查覆盖面进行有效的系统开发，这可能会提高商业产品和公共服务的安全性和有效性，并提高整体经济竞争力。该项目引入的创新教育范式将验证嵌入到常规工程作业中，使学生能够做出正确的判断，并成为工程领域的有效决策者。中学生参与了一个电脑游戏，该游戏教授计算之外的概念多样性。通过认识到没有单一的答案存在的工程问题，学生拥抱工程解决方案的潜在多样性，并专注于了解先验知识，心理模型，偏好和团队成员的态度如何塑造他们的解决方案的意义。这个CAREER项目的研究目标是研究系统工程验证活动中工程师信念的形成，并确定更好地使工程师能够在信息工件中捕获他们信念的模式和方法。工程师的信念如何形成的解释模型将通过结合贝叶斯分析，传统和动态的信念图，决策理论与专业工程师和信息工件的经验观察（例如，核查计划和报告）。认知映射将被用来引出信念结构，贝叶斯分析将被用来建模工程师的信念，和信用集将被用来解释不同的工程师的信念。 这个项目的教育目标是帮助学生认识和接受工程问题中正确性的主观（但非任意）性质。新的教学方法，两个关键人群的工程将实施和评估。一个数字游戏，以模拟现实世界的背景下，数学应用题将被用来寻求和捕捉概念的多样性，出现时，中学生合作解决问题。本科生将接触到开放式的问题，促进解决方案验证迭代方法，从他们的新生到高年级的几门课程的工程。总体而言，该项目将有助于确定工程师的信念和信念结构对验证策略和验证证据评估的影响。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

In systems engineering, verification is the process of building confidence in the proper functioning of a system. Although it is a key activity in engineering, the fundamentals of how engineers utilize objective data to inform inherently subjective judgments about the system during verification is not well understood. This Faculty Early Career Development Program (CAREER) award supports fundamental research into this process and education efforts geared towards improving understanding about and skills for verification activities. The research focus is on modeling the information structures by which engineers build confidence in a system's performance. This knowledge can lead to improved verification strategies and more rigorous ways to evaluate verification data. Hence, the research activities can bring about a potentially transformative paradigm for the practice of systems engineering. Improved verification strategies benefit society by enabling efficient system development with a more robust verification coverage, which may lead to increased safety and efficacy of commercial products and public services and greater overall economic competitiveness. The innovative education paradigm introduced in this project embeds verification into regular engineering assignments to prepare students to exercise sound judgement and be effective decision makers in engineering. Middle school students are engaged with a computer game that teaches conceptual diversity beyond calculations. By realizing that no single answer exists for an engineering problem, students embrace the potential diversity in engineering solutions and focus on understanding how prior knowledge, mental models, preferences, and team member attitudes shape the meaningfulness of their solutions. The research objective of this CAREER project is to study the formation of engineers' beliefs during systems engineering verification activities and to identify patterns and heuristics that better enable engineers to capture their beliefs in information artifacts. Explanatory models of how engineers' beliefs form will be established by combining Bayesian analyses, traditional and dynamic belief graphs, and decision theory with empirical observations of professional engineers and information artifacts (e.g., verification plans and reports). Cognitive mapping will be used to elicit belief structures, Bayesian analysis will be employed to model engineers' beliefs, and Credal sets will be used to account for differing engineers' beliefs. The educational objective of this project is to help students recognize and embrace the subjective (yet non-arbitrary) nature of correctness in engineering problems. Novel instructional approaches to two key populations for engineering will be implemented and assessed. A digital game to model real-world contexts as math word problems will be used to seek and capture the conceptual diversity that emerges when middle school students cooperate while solving problems. Undergraduate students will be exposed to open-ended problems that foster a solution-verification iterative approach to engineering in several courses from their freshmen to senior years. Overall, this project will contribute to determining the effects of engineers' beliefs and belief structures on verification strategies and the evaluation of verification evidence.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.