EAGER/Collaborative Research: Demonstrating the Importance of Research Setting Representativeness in Systems Engineering and Design Research

EARLY概念探索性研究资助（EAGER）资助支持基础研究，以确定研究环境是否足以代表现实世界的系统工程和设计情况。设计复杂的工程系统，如关键基础设施，资本密集型项目和国防系统，涉及由数百至数千人组成的大型设计团队，时间跨度长达数十年。该项目解决了使用更简单，持续时间更短，更可控的研究设置来理解这种复杂工程系统的设计的基本问题。本项目的研究目标是探索研究设置代表性的标准的创建和验证，以评估来自一个研究设置的结果是否可以扩展到其他研究设置。如果成功的话，这项工作将改变社区的研究，通过严格的方法论基础来评估研究环境在多大程度上以及在哪些方面代表了他们的真实的同行，从而实现更有效的知识聚合。该基金会将丰富系统工程和设计社区内的经验和理论驱动的研究，通过提高跨不同的方法论视角的研究成果的有效性评估。作为该项目的一个重要组成部分，将通过在社区专题会议上介绍情况和举办讲习班，向各种受众传播成果。传播和社区建设活动将促进围绕需要就从不同研究环境和方法学途径获得的知识种类制定共同标准和达成共识的持续对话。这种高风险高回报工作的核心智力贡献是确定研究设置代表性的维度，了解它们如何影响研究的设计和执行，并展示一种方法来产生关于代表性的见解。两项试点研究将探讨航天器设计过程中代表性的不同方面。第一项试点研究将比较真实的航天器设计过程与一个并行设计环境，其中一个较小的团队在较短的时间尺度上工作。第二个试点研究将比较一个完全抽象的功能优化游戏，一个更有代表性的游戏，模型的背景和技术依赖的航天器在优化问题。试点研究将证明研究的可行性，研究设置的代表性对设计师行为的影响。研究设置的代表性可能无法很好地表征，这是本项目的主要风险因素。然而，潜在的回报同样很高。如果成功，该项目的结果有望为多方法，多尺度研究提供基础，使研究人员能够相互解释，评估和建立？的结果，在不同的研究方法。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This EArly-concept Grant for Exploratory Research (EAGER) grant supports fundamental research into the factors that determine whether a research setting is sufficiently representative of real-world systems engineering and design situations. Designing complex engineering systems such as critical infrastructure, capital-intensive projects, and national defense systems involves large design teams consisting of hundreds to thousands of individuals over timescales reaching into decades. This project addresses the fundamental issue of understanding the design of such complex engineering systems using simpler, shorter-duration, and more controlled research settings. The research objective of this project is to explore the creation and validation of standards for research setting representativeness to assess whether and the extent to which findings from one research setting can be extend to others. If successful, this work will transform research in the community by enabling more efficient knowledge aggregation through a rigorous methodological foundation for assessing how well, and in what dimensions, research settings represent their real counterparts. This foundation will enrich empirical and theory-driven research within the systems engineering and design community by improving validity assessment of research outcomes across diverse methodological perspectives. As a crucial component of this project, results will be disseminated to diverse audience through presentations and workshops at community-specific conferences. Dissemination and community-building activities will foster an ongoing conversation around the need for shared standards and consensus for the kinds of knowledge to be gained from diverse research settings and methodological approaches. The core intellectual contributions of this high-risk high-reward work are to identify dimensions of research setting representativeness, understand how they impact design and execution of research studies, and demonstrate a methodology to generate insights regarding representativeness. Two pilot studies will probe varying dimensions of representativeness for a spacecraft design process. The first pilot study will compare the real spacecraft design process to a concurrent design environment with a smaller team working on a shorter timescale. The second pilot study will compare a fully abstracted function optimization game to a more representative game that models the context and technical dependencies of a spacecraft in the optimization problem. The pilot studies will demonstrate the viability of studying the effects of research setting representativeness on designer behaviors. The potential exists that research setting representativeness is impractical to characterize well, which is a principal risk factor for this project. However, the potential reward is equally high. If successful, results from this project are expected to provide the foundation of multi-method, multi-scale research, allowing researchers to interpret, evaluate, and build on each other?s results across different research methodologies. This would transform the research community and enable new discoveries that far exceed current research capabilities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.