Process Data for Modern Educational Assessment and Learning

该研究项目将使用过程数据来开发教育评估工具以及测试和学习的适应性设计。随着计算机测试的日益普及，人们收集了各种各样的行为数据。该项目将侧重于开发进行准确评估和提供有效的个性化学习材料/干预措施的方法。这些方法将基于计算机测试日志文件中收集的过程数据。要讨论的具体主题包括通过统计学习方法分析过程数据，开发基于过程数据的评估，以及通过过程数据进行适应性学习。这项研究的结果将提供一个更深入的了解学生的行为和认知过程在一个日益由基于技术的互动和交流所定义的环境。将提供提高物品质量的指导方针，重点关注更具创新性的物品类型，例如基于场景和基于模拟的环境中的物品。这项研究的结果将有利于指导和干预计划，旨在帮助学生在学术环境。此外，将开发开源软件，研究生将参与研究的进行。最近大规模的基于计算机的评估开发了一些互动式问题解决项目。这个研究项目将开发分析这些新项目的工具。研究人员将集中研究现代计算机评估和在线学习中非常具有挑战性的几个方面。具体而言，他们将关注以下主题：1)通过统计学习技术了解学生的认知过程，从过程数据中提取信息；2)利用过程数据改进现有的评价工具；3)将过程数据中的信息整合到在线适应/个性化学习中。分析将结合来自教育研究和统计学习的技术和概念。提出的模型将结合潜在变量建模和深度学习技术进行过程数据分析。研究人员将采用自然语言处理的建模和分割技术的最新进展。适应性学习将通过强化学习框架进行研究。此外，优化算法将通过数值方法的最新进展来开发。该奖项由MMS项目和联邦统计机构联盟支持。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This research project will use process data to develop tools for educational assessment and adaptive design for testing and learning. With the increasing use of computer-based testing, a variety of behavioral data have been collected. This project will focus on developing methods to conduct accurate assessments and deliver effective personalized learning materials/interventions. These methods will be based on process data collected in log files of computer-based tests. Specific topics to be addressed include the analysis of process data via statistical learning methods, development of process-data-based assessments, and adaptive learning through process data. The results of this research will provide a deeper understanding of students' behaviors and cognitive processes in an environment increasingly defined by technology-based interaction and communication. Guidelines to improve item quality will be provided, with a focus on more innovative item types such as those in scenario-based and simulation-based environments. The results of this research will benefit instruction and intervention programs designed to help students in academic environments. In addition, open-source software will be developed, and graduate students will be involved in the conduct of the research.Recent large-scale computer-based assessments have developed a number of interactive problem-solving items. This research project will develop tools for the analysis of these new items. The investigators will concentrate on several aspects that are very challenging in modern computer-based assessment and online learning. Specifically, they will focus on the following topics: 1) understanding students' cognitive processes by means of statistical learning techniques, extracting information from process data; 2) improving current assessment tools by means of process data; and 3) incorporating information in process data to online adaptive/personalized learning. The analysis will combine techniques and concepts from education research and statistical learning. The proposed models will combine latent variable modeling and deep learning techniques for process data analysis. The investigators will employ recent advances in modeling and segmenting techniques for natural language processing. Adaptive learning will be studied through a reinforcement learning framework. In addition, optimization algorithms will be developed by means of recent advances in numerical methods. This award is supported by the MMS Program and a consortium of Federal statistical agencies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.