Process Data for Modern Educational Assessment and Learning

这项研究项目将使用过程数据来开发教育评估工具以及测试和学习的适应性设计。随着计算机测试的使用越来越多，各种行为数据也被收集起来。该项目将侧重于制定进行准确评估和提供有效的个性化学习材料/干预措施的方法。这些方法将以计算机考试日志文件中收集的过程数据为基础。要讨论的具体主题包括通过统计学习方法分析过程数据、开发基于过程数据的评估以及通过过程数据进行适应性学习。这项研究的结果将有助于更深入地了解学生在以技术为基础的互动和交流日益定义的环境中的行为和认知过程。将提供改进项目质量的指导方针，重点是更具创新性的项目类型，如基于情景和基于模拟的环境中的项目类型。这项研究的结果将有益于旨在帮助学术环境中的学生的指导和干预计划。此外，将开发开放源码软件，研究生将参与研究。最近的大规模计算机评估开发了一些交互式问题解决项目。这一研究项目将开发分析这些新项目的工具。调查人员将专注于现代基于计算机的评估和在线学习中非常具有挑战性的几个方面。具体地说，他们将关注以下主题：1)通过统计学习技术了解学生的认知过程，从过程数据中提取信息；2)通过过程数据改进现有的评估工具；以及3)将过程数据中的信息纳入在线适应性/个性化学习。分析将结合教育、研究和统计学习的技术和概念。提出的模型将结合潜变量建模和深度学习技术进行过程数据分析。研究人员将采用自然语言处理的建模和分段技术方面的最新进展。自适应学习将通过强化学习框架进行研究。此外，还将利用数值方法的最新进展来开发优化算法。这一奖项由彩信计划和一个联邦统计机构联盟支持。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This research project will use process data to develop tools for educational assessment and adaptive design for testing and learning. With the increasing use of computer-based testing, a variety of behavioral data have been collected. This project will focus on developing methods to conduct accurate assessments and deliver effective personalized learning materials/interventions. These methods will be based on process data collected in log files of computer-based tests. Specific topics to be addressed include the analysis of process data via statistical learning methods, development of process-data-based assessments, and adaptive learning through process data. The results of this research will provide a deeper understanding of students' behaviors and cognitive processes in an environment increasingly defined by technology-based interaction and communication. Guidelines to improve item quality will be provided, with a focus on more innovative item types such as those in scenario-based and simulation-based environments. The results of this research will benefit instruction and intervention programs designed to help students in academic environments. In addition, open-source software will be developed, and graduate students will be involved in the conduct of the research.Recent large-scale computer-based assessments have developed a number of interactive problem-solving items. This research project will develop tools for the analysis of these new items. The investigators will concentrate on several aspects that are very challenging in modern computer-based assessment and online learning. Specifically, they will focus on the following topics: 1) understanding students' cognitive processes by means of statistical learning techniques, extracting information from process data; 2) improving current assessment tools by means of process data; and 3) incorporating information in process data to online adaptive/personalized learning. The analysis will combine techniques and concepts from education research and statistical learning. The proposed models will combine latent variable modeling and deep learning techniques for process data analysis. The investigators will employ recent advances in modeling and segmenting techniques for natural language processing. Adaptive learning will be studied through a reinforcement learning framework. In addition, optimization algorithms will be developed by means of recent advances in numerical methods. This award is supported by the MMS Program and a consortium of Federal statistical agencies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.