Supporting Instructional Decision Making: The Potential of An Automatically Scored Three-dimensional Assessment System

支持教学决策：自动评分三维评估系统的潜力

• 批准号: 2101166
• 负责人: Yue Yin
• 金额: $ 59.66万
• 依托单位: University of Illinois at Chicago
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-01 至 2025-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2101166&HistoricalAwards=false
• 关键词: Supporting; Instructional; Decision; Making; Potential

This project will study the utility of a machine learning-based assessment system for supporting middle school science teachers in making instructional decisions based on automatically generated student reports (AutoRs). The assessments target three-dimensional (3D) science learning by requiring students to integrate scientific practices, crosscutting concepts, and disciplinary core ideas to make sense of phenomena or solve complex problems. Led by collaborators from University of Georgia, Michigan State University, University of Illinois at Chicago, and WestEd, the project team will develop computer scoring algorithms, a suite of AutoRs, and an array of pedagogical content knowledge supports (PCKSs). These products will assist middle school science teachers in the use of 3D assessments, making informative instructional changes, and improve students’ 3D learning. The project will generate knowledge about teachers’ uses of 3D assessments and examine the potential of automatically scored 3D assessments. The project will achieve the research goals using a mixed-methods design in three phases. Phase I: Develop AutoRs. Machine scoring models for the 3D assessment tasks will be developed using existing data. To support teachers’ interpretation and use of automatic scores, the project team will develop AutoRs and examine how teachers make use of these initial reports. Based on observations and feedback from teachers, AutoRs will be refined using an iterative procedure so that teachers can use them with more efficiency and productivity. Phase II: Develop and test PCKSs. Findings from Phase I, the literature, and interviews with experienced teachers will be employed to develop PCKSs. The project will provide professional learning with teachers on how to use the AutoRs and PCKSs. The project will research how teachers use AutoRs and PCKSs to make instructional decisions. The findings will be used to refine the PCKSs. Phase III: Classroom implementation. In this phase a study will be conducted with a new group of teachers to explore the effectiveness and usability of AutoRs and PCKSs in terms of supporting teachers’ instructional decisions and students’ 3D learning. This project will create knowledge about and formulate a theory of how teachers interpret and attend to students’ performance on 3D assessments, providing critical information on how to support teachers’ responsive instructional decision making. The collaborative team will widely disseminate various products, such as 3D assessment scoring algorithms, AutoRs, PCKSs, and the corresponding professional development programs, and publications to facilitate 3D instruction and learning.The Discovery Research preK-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目将研究基于机器学习的评估系统的效用，以支持中学科学教师根据自动生成的学生报告（AutoRs）做出教学决策。该评估以三维（3D）科学学习为目标，要求学生整合科学实践、横切概念和学科核心思想，以理解现象或解决复杂问题。在来自佐治亚大学、密歇根州立大学、伊利诺伊大学芝加哥分校和WestEd的合作者的领导下，项目团队将开发计算机评分算法、一套autor和一系列教学内容知识支持（PCKSs）。这些产品将帮助中学科学教师使用3D评估，进行翔实的教学改变，并改善学生的3D学习。该项目将产生有关教师使用3D评估的知识，并研究自动评分3D评估的潜力。该项目将采用混合方法设计，分三个阶段实现研究目标。阶段1：开发AutoRs。3D评估任务的机器评分模型将使用现有数据开发。为了支持教师对自动评分的解释和使用，项目团队将开发AutoRs，并检查教师如何使用这些初始报告。根据教师的观察和反馈，AutoRs将通过迭代过程进行改进，以便教师能够更有效地使用它们。第二阶段：开发和测试pcks。第一阶段的研究结果、文献和对有经验的教师的访谈将被用于发展PCKSs。该项目将为教师提供如何使用autor和PCKSs的专业学习。该项目将研究教师如何使用autor和PCKSs来做出教学决策。研究结果将用于改进PCKSs。第三阶段：课堂实施。在这一阶段，我们将与一组新的教师进行一项研究，探讨AutoRs和PCKSs在支持教师教学决策和学生3D学习方面的有效性和可用性。该项目将创建有关教师如何解释和关注学生在3D评估中的表现的知识和理论，为如何支持教师响应性教学决策提供关键信息。合作团队将广泛传播各种产品，如3D评估评分算法、AutoRs、PCKSs以及相应的专业发展计划和出版物，以促进3D教学和学习。探索研究preK-12项目（DRK-12）旨在通过研究和开发创新资源、模型和工具，显著提高preK-12学生和教师对科学、技术、工程和数学（STEM）的学习和教学。DRK-12计划中的项目建立在STEM教育的基础研究和先前的研究和开发工作的基础上，为拟议的项目提供了理论和实证依据。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。