EAGER: Collaborative Research: Changing the Paradigm: Developing a Framework for Secondary Analysis of EER Datasets

为了帮助培养国家的工程人才，美国国家科学基金会在过去的二十年里投入了大量的公共资金用于工程教育研究。这项投资通过测试和分享基于研究的实践，显著改善了许多大学的课程和项目，这些实践促进了主动学习，提高了学生的积极性和参与度，使领域多样化，并使学生更好地为工作做好准备。与此同时，投资通常集中在研究人员收集新数据上，导致数百个数据集仍未得到充分开发。这些现有的数据集具有巨大的分析潜力，甚至可以以新的方式进行组合，以进一步支持我们在招聘、教学和培养工程学生方面的大规模变革，以应对21世纪的需求和挑战。然而，目前，工程教育研究人员没有有效的方法来共享和分析原始项目之外的数据。因此，这些数据集的全部潜力仍未得到开发。该项目将通过开发和推广一种可行的方法来解决这一差距，使研究人员能够利用目前可用的丰富数据。这样做，将同时改善全国的工程教育，提高公共资金的投资回报率。该项目将把经验丰富的研究人员与刚刚开始职业生涯的研究人员聚集在一起，确定共享和重用数据的主要障碍，制定克服这些障碍的策略和实践，并进行一系列测试用例，展示如何将这些策略和实践付诸行动。研究结果将有助于创造一种范式转变，将美国工程教育的研究和实践提升到一个新的水平，并刺激那种使美国工程劳动力保持在全球市场前沿所需的巨大变化。改变一次性数据收集的模式是一个高风险的命题，需要可操作的、经过验证的有效的、合乎道德的数据共享实践，以及充分的激励机制来共享和使用现有数据。为此，本提案汇集了一个专家团队，以克服数据共享方面的实质性障碍，并建立一个框架来指导工程教育研究中的二级分析。特别是，我们将汇集知名学者和新兴学者，提供一个经过测试的框架，概述正式和非正式共享数据、公开数据集、结合不同研究的数据、对定性和定量数据进行二次分析、发布和共享结果、获得所需资金以及确保工作在该领域得到重视的方法学最佳实践。为了创建这个框架，研究小组将在两年内举行一系列六次研讨会。在第一年，我们将把来自全国各机构的备受尊敬、经验丰富的研究人员与新研究人员聚集在一起，创建数据共享和数据重用的初步框架。在第二年，我们将在两个现有数据集上测试和完善该框架。我们将从更广泛的社区征集数据集，并邀请学者团队与原始研究人员进行对话，对这些数据集进行二次分析。重要的是，在选择数据集和二次分析方法时，我们将强调方法的多样性，以确保框架广泛适用。结果将是一份框架文件，其中将包括一套经过测试的指导方针，用于工程教育研究中的数据共享和二次分析，并通过期刊和研讨会分发，以促进广泛采用。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

To help develop the nation’s engineering workforce, the National Science Foundation has invested substantial public funding in engineering education research over the past twenty years. This investment has helped markedly improve courses and programs at many universities by testing and sharing research-based practices that promote active learning, increase student motivation and engagement, diversify the field, and better prepare students for work. At the same time, the investment has typically focused on researchers collecting new data, resulting in hundreds of data sets that remain underexplored. These existing data sets have significant potential to be analyzed and even combined in new ways to further support large-scale changes in how we recruit, teach, and prepare engineering students for the demands and challenges of the 21st century. Currently, however, engineering education researchers do not have productive and effective ways for sharing and analyzing data beyond the original project. Thus the full potential of these data sets remains untapped. This project will address that gap by developing and promoting a viable approach that will enable researchers to leverage the rich data currently available. In doing so, it will simultaneously improve engineering education nationally and increase the return on investment of public funds. The project will bring experienced researchers together with those just beginning their careers to identify the major roadblocks to sharing and re-using data, develop strategies and practices for overcoming those roadblocks, and conduct a series of test cases that demonstrate how to put those strategies and practices into action. The results will help create a paradigm shift that can move both the study and the practice of engineering education in the U.S. to a new level and spur the kind of sea changes needed to keep the nation’s engineering workforce at the forefront of the global marketplace.Changing the paradigm of single-use data collection is a high-risk proposition that requires actionable, proven practices for effective, ethical data sharing, coupled with sufficient incentives to both share and use existing data. To that end, this proposal draws together a team of experts to overcome substantial obstacles in data sharing and build a framework to guide secondary analysis in engineering education research. In particular, we will bring together established and emerging scholars to deliver a tested framework that outlines methodological best practices for formally and informally sharing data, making data sets public, combining data from different studies, performing secondary analyses of both qualitative and quantitative data, publishing and sharing the results, securing the needed funding, and ensuring that the work is valued in the field. To create this framework, the research team will hold a series of six workshops over two years. In the first year, we will bring highly respected, experienced researchers from institutions across the country together with newer researchers to create the initial framework for data sharing and data re-use. In the second year, we will test and refine that framework on two existing data sets. We will solicit data sets from the wider community, and invite teams of scholars to conduct secondary analysis on those data sets, in conversation with the original researchers. Importantly, in selecting both the data sets and the approaches to secondary analysis, we will emphasize methodological diversity to ensure that the framework is widely applicable. The outcome will be a framework document that will comprise a set of tested guidelines for data sharing and secondary analysis in engineering education research, distributed through both journals and workshops to promote widespread adoption.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.