EAGER: Collaborative Research: Changing the Paradigm: Developing a Framework for Secondary Analysis of EER Datasets

为了帮助发展国家的工程劳动力，国家科学基金会在过去的二十年里在工程教育研究上投入了大量的公共资金。这项投资通过测试和分享基于研究的实践来显著改进许多大学的课程和项目，这些实践促进了主动学习，提高了学生的积极性和参与度，使领域多样化，并为学生的工作做好了更好的准备。与此同时，投资通常集中在研究人员收集新数据上，导致数百个数据集仍未得到充分开发。这些现有的数据集具有巨大的潜力，可以通过新的方式进行分析甚至组合，以进一步支持我们如何招收、教授和准备工程专业学生以应对21世纪的需求和挑战的大规模变化。然而，目前，工程教育研究人员没有有效的方法来共享和分析原始项目以外的数据。因此，这些数据集的全部潜力仍未得到开发。该项目将通过开发和推广一种可行的方法来解决这一差距，该方法将使研究人员能够利用现有的丰富数据。通过这样做，将同时改善全国的工程教育，并增加公共资金的投资回报。该项目将把有经验的研究人员和那些刚刚开始其职业生涯的研究人员聚集在一起，确定共享和重复使用数据的主要障碍，制定克服这些障碍的战略和做法，并进行一系列测试案例，展示如何将这些战略和做法付诸实施。这一结果将有助于创造一种范式转变，将美国工程教育的研究和实践推向一个新的水平，并刺激保持美国工程劳动力处于全球市场前沿所需的那种巨变。改变一次性数据收集范式是一个高风险的提议，需要有效、合乎道德的数据共享的可操作、经过验证的实践，以及充分的激励措施来共享和使用现有数据。为此，这项建议汇集了一个专家团队，以克服数据共享方面的重大障碍，并建立一个框架来指导工程教育研究中的二次分析。特别是，我们将召集老牌和新兴学者，提供经过测试的框架，概述正式和非正式共享数据、公开数据集、合并不同研究的数据、对定性和定量数据进行二次分析、发布和共享结果、获得所需资金以及确保工作在实地得到重视的方法论最佳做法。为了建立这一框架，研究小组将在两年内举办一系列六次研讨会。在第一年，我们将把来自全国各地机构的备受尊敬、经验丰富的研究人员与较新的研究人员聚集在一起，创建数据共享和数据重用的初步框架。在第二年，我们将在两个现有的数据集上测试和完善该框架。我们将从更广泛的社区征求数据集，并邀请学者团队在与原始研究人员的对话中对这些数据集进行二次分析。重要的是，在选择数据集和二次分析方法时，我们将强调方法多样性，以确保框架广泛适用。结果将是一份框架文件，其中将包括一套经过测试的指导方针，用于工程教育研究中的数据共享和二次分析，通过期刊和研讨会分发，以促进广泛采用。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

To help develop the nation’s engineering workforce, the National Science Foundation has invested substantial public funding in engineering education research over the past twenty years. This investment has helped markedly improve courses and programs at many universities by testing and sharing research-based practices that promote active learning, increase student motivation and engagement, diversify the field, and better prepare students for work. At the same time, the investment has typically focused on researchers collecting new data, resulting in hundreds of data sets that remain underexplored. These existing data sets have significant potential to be analyzed and even combined in new ways to further support large-scale changes in how we recruit, teach, and prepare engineering students for the demands and challenges of the 21st century. Currently, however, engineering education researchers do not have productive and effective ways for sharing and analyzing data beyond the original project. Thus the full potential of these data sets remains untapped. This project will address that gap by developing and promoting a viable approach that will enable researchers to leverage the rich data currently available. In doing so, it will simultaneously improve engineering education nationally and increase the return on investment of public funds. The project will bring experienced researchers together with those just beginning their careers to identify the major roadblocks to sharing and re-using data, develop strategies and practices for overcoming those roadblocks, and conduct a series of test cases that demonstrate how to put those strategies and practices into action. The results will help create a paradigm shift that can move both the study and the practice of engineering education in the U.S. to a new level and spur the kind of sea changes needed to keep the nation’s engineering workforce at the forefront of the global marketplace.Changing the paradigm of single-use data collection is a high-risk proposition that requires actionable, proven practices for effective, ethical data sharing, coupled with sufficient incentives to both share and use existing data. To that end, this proposal draws together a team of experts to overcome substantial obstacles in data sharing and build a framework to guide secondary analysis in engineering education research. In particular, we will bring together established and emerging scholars to deliver a tested framework that outlines methodological best practices for formally and informally sharing data, making data sets public, combining data from different studies, performing secondary analyses of both qualitative and quantitative data, publishing and sharing the results, securing the needed funding, and ensuring that the work is valued in the field. To create this framework, the research team will hold a series of six workshops over two years. In the first year, we will bring highly respected, experienced researchers from institutions across the country together with newer researchers to create the initial framework for data sharing and data re-use. In the second year, we will test and refine that framework on two existing data sets. We will solicit data sets from the wider community, and invite teams of scholars to conduct secondary analysis on those data sets, in conversation with the original researchers. Importantly, in selecting both the data sets and the approaches to secondary analysis, we will emphasize methodological diversity to ensure that the framework is widely applicable. The outcome will be a framework document that will comprise a set of tested guidelines for data sharing and secondary analysis in engineering education research, distributed through both journals and workshops to promote widespread adoption.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.