Process Data for Modern Educational Assessment and Learning

现代教育评估和学习的处理数据

• 批准号: 2119938
• 负责人: Jingchen Liu
• 金额: $ 37.5万
• 依托单位: Columbia University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-02-01 至 2025-01-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2119938&HistoricalAwards=false
• 关键词: Process; Data; Modern; Educational; Assessment

This research project will use process data to develop tools for educational assessment and adaptive design for testing and learning. With the increasing use of computer-based testing, a variety of behavioral data have been collected. This project will focus on developing methods to conduct accurate assessments and deliver effective personalized learning materials/interventions. These methods will be based on process data collected in log files of computer-based tests. Specific topics to be addressed include the analysis of process data via statistical learning methods, development of process-data-based assessments, and adaptive learning through process data. The results of this research will provide a deeper understanding of students' behaviors and cognitive processes in an environment increasingly defined by technology-based interaction and communication. Guidelines to improve item quality will be provided, with a focus on more innovative item types such as those in scenario-based and simulation-based environments. The results of this research will benefit instruction and intervention programs designed to help students in academic environments. In addition, open-source software will be developed, and graduate students will be involved in the conduct of the research.Recent large-scale computer-based assessments have developed a number of interactive problem-solving items. This research project will develop tools for the analysis of these new items. The investigators will concentrate on several aspects that are very challenging in modern computer-based assessment and online learning. Specifically, they will focus on the following topics: 1) understanding students' cognitive processes by means of statistical learning techniques, extracting information from process data; 2) improving current assessment tools by means of process data; and 3) incorporating information in process data to online adaptive/personalized learning. The analysis will combine techniques and concepts from education research and statistical learning. The proposed models will combine latent variable modeling and deep learning techniques for process data analysis. The investigators will employ recent advances in modeling and segmenting techniques for natural language processing. Adaptive learning will be studied through a reinforcement learning framework. In addition, optimization algorithms will be developed by means of recent advances in numerical methods. This award is supported by the MMS Program and a consortium of Federal statistical agencies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该研究项目将使用过程数据开发教育评估工具和测试和学习的自适应设计。随着越来越多的使用基于计算机的测试，各种各样的行为数据已被收集。该项目将侧重于制定进行准确评估和提供有效的个性化学习材料/干预措施的方法。这些方法将基于在基于计算机的测试的日志文件中收集的过程数据。要解决的具体问题包括通过统计学习方法分析过程数据，开发基于过程数据的评估，以及通过过程数据进行自适应学习。这项研究的结果将提供一个更深入的了解学生的行为和认知过程中的环境越来越多地定义为基于技术的互动和沟通。将提供提高物品质量的指南，重点关注更具创新性的物品类型，例如基于场景和基于模拟的环境中的物品类型。这项研究的结果将有利于指导和干预计划，旨在帮助学生在学术环境。此外，还将开发开放源码软件，研究生将参与研究工作，最近的大规模计算机评估开发了一些互动式解决问题项目。该研究项目将开发分析这些新项目的工具。调查人员将集中在几个方面，这是非常具有挑战性的现代计算机为基础的评估和在线学习。具体而言，他们将专注于以下主题：1）通过统计学习技术了解学生的认知过程，从过程数据中提取信息; 2）通过过程数据改进当前的评估工具; 3）将过程数据中的信息纳入在线自适应/个性化学习。分析将结合联合收割机的技术和概念，从教育研究和统计学习。所提出的模型将结合联合收割机潜变量建模和深度学习技术进行过程数据分析。研究人员将采用自然语言处理建模和分割技术的最新进展。自适应学习将通过强化学习框架进行研究。此外，优化算法将通过数值方法的最新进展来开发。该奖项由MMS计划和联邦统计机构联盟支持。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Statistical Applications to Cognitive Diagnostic Testing

认知诊断测试的统计应用

• DOI: 10.1146/annurev-statistics-033021-111803
• 发表时间: 2023
• 期刊: Annual Review of Statistics and Its Application
• 影响因子: 7.9
• 作者: Zhang, Susu;Liu, Jingchen;Ying, Zhiliang
• 通讯作者: Ying, Zhiliang

ProcData: An R Package for Process Data Analysis

• DOI: 10.1007/s11336-021-09798-7
• 发表时间: 2020-06
• 期刊: Psychometrika
• 影响因子: 3
• 作者: Xueying Tang;Susu Zhang;Zhi Wang;Jingchen Liu;Z. Ying

Accurate Assessment via Process Data

• DOI: 10.1007/s11336-022-09880-8
• 发表时间: 2021-03
• 期刊: Psychometrika
• 影响因子: 3
• 作者: Susu Zhang;Zhi Wang;Jitong Qi;Jingchen Liu;Z. Ying

Item Response Theory -- A Statistical Framework for Educational and Psychological Measurement

• 发表时间: 2021-08
• 期刊: Psychometrika
• 影响因子: 3
• 作者: Yunxiao Chen;Xiaoou Li;Jingchen Liu;Z. Ying