Collaborative Research: Large-Scale Research on Engineering Design Based on Big Learner Data Logged by a CAD Tool

参与机构：协和联盟(牵头)普渡大学核心领域(S)：STEM学习/STEM学习环境项目描述实践科学是K-12工程教育最重要的目标之一，它现在是下一代科学标准的一部分。尽管以前的研究表明，工程设计是促进科学学习的一种有效的教学方法，但也有人担心，在设计项目中，“设计-科学鸿沟”会导致科学学习失败。这个项目正在深入研究大量的过程数据，以系统地识别设计过程中给学生应用科学带来困难的瓶颈。从印第安纳州和马萨诸塞州的3000多名学生那里收集了大量的学习者数据集，通过一种独特的CAD工具自动、不引人注目地记录学生设计过程，该工具支持使用热力学和传热学概念设计节能建筑。大数据集--包括学生行为、实验结果、电子笔记和设计人工制品的细粒度信息--被用来重建每个学生的整个学习轨迹。正在开发和应用强大的过程分析(例如，时间序列分析和关联规则挖掘)，以揭示学生群体和知识领域的模式和趋势。通过将这些大型数据集与前测/后测和人口统计数据相结合，该项目回答了以下研究问题：RQ1：学生设计行为的常见模式是什么，它们与先前的知识、项目持续时间、设计表现、学习结果和人口统计因素有何关联？RQ2：学生如何加深对工程设计项目中涉及的科学概念的理解？RQ3：学生使用科学实验进行设计选择的频率和深度有多大？这个为期五年的项目从第一年和第二年的六项小规模研究开始，通过与课堂观察、专家评估和学生访谈进行比较来校准过程分析。然后，过程分析将在文献元分析的基础上，通过使用知情设计教学和学习矩阵来验证研究方法。BROADER意义项目的规模将允许更多地代表学生多样性，这在小规模研究中是不容易实现的。该项目正在通过研究最复杂的STEM实践之一--工程设计，为新兴的教育数据挖掘和学习分析领域做出贡献。计算机辅助设计数据具有IBM定义的大数据的所有四个特征。大数据有可能产生直接的、可测量的证据，证明在统计上具有重大意义的学习。自动化使这一研究方法具有高度的可扩展性，自动化过程分析正在为构建用于教学工程设计的适应性和预测性软件铺平道路。作为该项目的副产品，编辑后的数据集将免费提供给任何有兴趣挖掘它们的研究人员。

PARTICIPATING INSTITUTIONS: Concord Consortium (Lead)Purdue UniversityCORE AREA(s): STEM Learning/STEM Learning EnvironmentsPROJECT DESCRIPTION Practicing science is one of the most important goals of K-12 engineering education, which is now part of the Next Generation Science Standards. Although previous research suggests that engineering design is an effective pedagogical approach to promoting science learning, there are concerns about the "design-science gap" that fails science learning in design projects. This project is delving into large quantities of process data to systematically identify bottlenecks in design processes that pose difficulties for students to apply science. Large learner datasets are being collected from over 3,000 students in Indiana and Massachusetts through automatic, unobtrusive logging of student design processes enabled by a unique CAD tool that supports the design of energy-efficient buildings using thermodynamics and heat transfer concepts. Large data sets - consisting of fine-grained information of student actions, experimentation results, electronic notes, and design artifacts - are used to reconstruct the entire learning trajectory of each individual student. Powerful process analytics (e.g., time series analysis and association rule mining) are being developed and applied to reveal patterns and trends across student groups and knowledge domains. Through a combination of these large data sets with pre/post-tests and demographic data, this project is answering the following research questions: RQ1: What are the common patterns of student design behaviors and how are they associated with prior knowledge, project duration, design performance, learning outcomes, and demographic factors? RQ2: How do students deepen their understanding of science concepts involved in engineering design projects? RQ3: How often and deeply do students use scientific experimentation to make a design choice? This five-year project is starting with six small-scale studies in years 1&2 to calibrate the process analytics by comparing with classroom observations, expert evaluations, and student interviews. The process analytics will then validate the research methodology by using the Informed Design Teaching and Learning Matrix, based on a meta-analysis of literature.BROADER SIGNIFICANCE The scale of the project will allow for greater representation of student diversity that is not readily attainable in small-scale studies. The project is contributing to the emerging fields of educational data mining and learning analytics through researching one of the most complex STEM practices -- engineering design. Computer Aided Design data possess all four characteristics of big data defined by IBM. The big data have the potential to yield direct, measurable evidence of learning at a statistically significant scale. Automation is making this research approach highly scalable and automatic process analytics is paving the road for building adaptive and predictive software for teaching engineering design. As a by-product of this project, the redacted datasets will be freely available to any researcher who is interested in mining them.