Collaborative Research: Large-Scale Research on Engineering Design Based on Big Learner Data Logged by a CAD Tool

参与机构：康科德财团（牵头）普渡大学核心领域：干学习/干学习环境项目描述实践科学是K-12工程教育的最重要的目标之一，现在是下一代科学标准的一部分。虽然以前的研究表明，工程设计是一种有效的教学方法，以促进科学学习，有关于“设计科学差距”，失败的科学学习设计项目的关注。该项目正在深入研究大量的过程数据，以系统地确定设计过程中的瓶颈，这些瓶颈给学生应用科学带来了困难。大型学习者数据集是从印第安纳州和马萨诸塞州的3,000多名学生中收集的，通过一个独特的CAD工具自动、不引人注目地记录学生的设计过程，该工具支持使用热力学和传热概念设计节能建筑。大型数据集-由学生行为，实验结果，电子笔记和设计工件的细粒度信息组成-用于重建每个学生的整个学习轨迹。强大的过程分析（例如，时间序列分析和关联规则挖掘），以揭示学生群体和知识领域的模式和趋势。通过结合这些大数据集与前/后测试和人口统计数据，该项目回答了以下研究问题：RQ 1：学生设计行为的常见模式是什么？它们如何与先验知识，项目持续时间，设计性能，学习成果和人口统计因素相关联？学生如何加深对工程设计项目中涉及的科学概念的理解？RQ 3：学生使用科学实验来做出设计选择的频率和深度如何？这个为期五年的项目是从6个小规模的研究在第1 - 2年校准过程分析通过比较课堂观察，专家评估和学生访谈。然后，过程分析将在文献荟萃分析的基础上，通过使用知情设计教学矩阵来验证研究方法。更广泛的意义该项目的规模将允许更大程度地代表学生的多样性，这在小规模研究中是不容易实现的。该项目通过研究最复杂的STEM实践之一-工程设计，为教育数据挖掘和学习分析的新兴领域做出贡献。计算机辅助设计数据具有IBM定义的大数据的所有四个特征。大数据有可能产生直接的，可测量的证据，以统计学上显著的规模进行学习。自动化使这种研究方法具有高度可扩展性，自动过程分析为构建用于教学工程设计的自适应和预测软件铺平了道路。作为该项目的副产品，编辑后的数据集将免费提供给任何有兴趣挖掘它们的研究人员。

PARTICIPATING INSTITUTIONS: Concord Consortium (Lead)Purdue UniversityCORE AREA(s): STEM Learning/STEM Learning EnvironmentsPROJECT DESCRIPTION Practicing science is one of the most important goals of K-12 engineering education, which is now part of the Next Generation Science Standards. Although previous research suggests that engineering design is an effective pedagogical approach to promoting science learning, there are concerns about the "design-science gap" that fails science learning in design projects. This project is delving into large quantities of process data to systematically identify bottlenecks in design processes that pose difficulties for students to apply science. Large learner datasets are being collected from over 3,000 students in Indiana and Massachusetts through automatic, unobtrusive logging of student design processes enabled by a unique CAD tool that supports the design of energy-efficient buildings using thermodynamics and heat transfer concepts. Large data sets - consisting of fine-grained information of student actions, experimentation results, electronic notes, and design artifacts - are used to reconstruct the entire learning trajectory of each individual student. Powerful process analytics (e.g., time series analysis and association rule mining) are being developed and applied to reveal patterns and trends across student groups and knowledge domains. Through a combination of these large data sets with pre/post-tests and demographic data, this project is answering the following research questions: RQ1: What are the common patterns of student design behaviors and how are they associated with prior knowledge, project duration, design performance, learning outcomes, and demographic factors? RQ2: How do students deepen their understanding of science concepts involved in engineering design projects? RQ3: How often and deeply do students use scientific experimentation to make a design choice? This five-year project is starting with six small-scale studies in years 1&2 to calibrate the process analytics by comparing with classroom observations, expert evaluations, and student interviews. The process analytics will then validate the research methodology by using the Informed Design Teaching and Learning Matrix, based on a meta-analysis of literature.BROADER SIGNIFICANCE The scale of the project will allow for greater representation of student diversity that is not readily attainable in small-scale studies. The project is contributing to the emerging fields of educational data mining and learning analytics through researching one of the most complex STEM practices -- engineering design. Computer Aided Design data possess all four characteristics of big data defined by IBM. The big data have the potential to yield direct, measurable evidence of learning at a statistically significant scale. Automation is making this research approach highly scalable and automatic process analytics is paving the road for building adaptive and predictive software for teaching engineering design. As a by-product of this project, the redacted datasets will be freely available to any researcher who is interested in mining them.