Collaborative Research: Research: Development and Validation of Learning Through Making Instrument (LMI)

创客空间是一种非正式的学习环境，通过迭代实践、协作、设计、构建、原型和管理促进多模式学习。它们在美国的大学校园里遍地开花，最初的创建成本在数千到数百万美元之间。虽然新兴数据表明创客空间可能对工程学习体验产生巨大影响，特别是在学习工程相关技能方面，但数据也表明，创客空间未能满足人人平等、开放、包容和欢迎的愿望。目前，很难确定一个创客空间的学习影响，也很难比较不同机构、不同学生群体的不同创客空间结构或创客空间。通过定性研究确定了大学创客空间中的学习模式和内容，本项目将创建并验证一个测量学生在创客空间中的学习的工具：通过制作学习调查工具。通过制作工具学习将为研究人员和教育工作者提供一个工具，以评估创客空间的教育价值，在不同的环境中进行比较，实验不同的方法，记录学习影响，并最终促进理解创客空间如何增强工程教育。该工具将帮助教育工作者了解可能与女性、有色人种学生、国际学生和第一代大学生障碍有关的学习差距，为教育工作者提供关键数据，以设计更具包容性的创客空间和相关课程，扩大参与，培养更具包容性的学习社区。“通过制作工具学习”将为大学生开发和验证，为创客空间学习的定量研究提供跳板，并将促进未来适用于K-12学生的类似工具的开发。本研究的目的是为衡量大学创客空间和类似环境中的学习进行调查。提案团队之前的工作“通过制作类型学来学习”将成为“通过制作乐器来学习”开发的部分基础。不同的工程专业学生的仪器开发和验证数据将出现在不同的大学，包括圣何塞州立大学、德克萨斯州立大学、德克萨斯农工大学、普渡大学、佐治亚理工学院和詹姆斯麦迪逊大学。入选的机构包括位于该国两个不同地区的两个拉美裔服务机构；一所综合性的、大西洋中部地区的、以本科为重点的大学，只提供普通工程学位；的大学之一;还有一所培养黑人和女性工程师的顶尖学校，该学校还开设双学位课程，为hbcu的学生提供入学机会。这些大学还提供了一组对比鲜明的创客空间。将产生一个初始的项目池，专家将评估项目池，并通过探索性因子分析和验证性因子分析寻求信度和结构效度的证据。《教育与心理测试标准》表明，评估的公平性与效度和信度一样重要，评估公平性将是本研究的关键部分。我们试图回答以下研究问题：测量结构规范和实际评估问题在多大程度上是一致的？在多大程度上，性别和少数群体之间存在测量不变性？各组之间的平均得分是否存在显著差异，表明某些组在创客空间中学习方式不同？该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Makerspaces are informal learning environments that facilitate multimodal learning through iterative practice, collaboration, design, building, prototype, and management. They have proliferated America’s college campuses, with initial creation costs in the thousands to millions of dollars. While emerging data demonstrate the tremendous impact makerspaces may create in the engineering learning experience especially for learning engineering-relevant skills, data also demonstrate that makerspaces fail to meet aspirations of being egalitarian, open, inclusive, and welcoming spaces for all. Currently, it is difficult to determine the learning impact of a makerspace and to compare different makerspaces structures or makerspaces at different institutions with different populations of students. Drawing upon qualitative research that identified the modes and contents of learning in university makerspaces, this project will create and validate an instrument that measures student learning in makerspaces: the Learning through Making survey Instrument. The Learning through Making Instrument will provide a tool for researchers and educators to assess the educational value of makerspaces, compare across different environments, experiment with different approaches, document the learning impacts, and ultimately facilitate understanding how makerspaces can enhance engineering education. This instrument will assist educators in understanding learning gaps that may be associated with obstacles for women, students of color, international students, and first-generation college students providing educators with data critical for designing more inclusive makerspaces and associated curriculum that broaden participation and cultivate more inclusive learning communities. The Learning through Making Instrument will be developed and validated for collegiate students providing a springboard for quantitative research on learning in makerspaces and will facilitate future development of similar instruments applicable for K-12 students.The research objective is to develop a survey for measuring learning in university makerspaces and similar environments. The proposal team’s prior work, The Learning through Making Typology will be a partial basis for development of the Learning through Making Instrument. Data for instrument development and validations with diverse engineering students will occur at very different universities, including San Jose State, Texas State, Texas A&M, Purdue University, Georgia Tech, and James Madison University. The selection includes two Hispanic-Serving Institutions in two different regions of the country; a comprehensive, mid-Atlantic, regional, undergraduate-focused university offering only a general engineering degree; research-focused universities; and a top school for graduating Black and women engineers that also has dual-degree programs providing access to students from HBCUs. These universities also provide a contrasting set of makerspaces. An initial pool of items will be generated, experts will evaluate the pool of items, and evidence of reliability along with structural validity will be sought by exploratory factor analysis and then confirmatory factor analysis. Standards for Educational and Psychological Testing indicate that fairness of assessments is just as crucial as validity and reliability, and fairness will be evaluated as a key part of this research. We seek to answer the following Research Questions: To what extent are measurement construct specifications and actual assessment questions aligned? To what extent are there measurement invariances between gender and minority groups? Are there significant differences in the mean scores between groups that indicate certain groups are learning differently in makerspaces?This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.