Statistical Learning for Innovative Assessment

这项研究项目将为现代认知评估开发统计方法。随着计算机测试的使用越来越多，各种高维的、复杂的结构化数据集被收集起来。该项目将侧重于复杂结构的大规模数据集的统计建模和推断。要讨论的具体主题包括过程数据分析、通过强化学习框架进行的自适应学习，以及为待开发的模型开发计算方法。这项研究的结果将提供对收集在技术丰富的交互任务中的复杂数据结构的更深层次的理解。该项目将阐明在客户-服务器星座和独立应用程序中在线提供的学习和评估环境中的项目。该项目将提供提高项目质量的指导方针，重点放在更具创新性的项目类型上，例如在基于情景和基于模拟的环境中评估学生在STEM领域的知识和技能。将向教育研究人员提供工具，以识别高维数据和序列数据中的模式。教学和干预项目的学生将从这项研究中受益，特别是在STEM领域，这些领域越来越多地被数字媒体和基于技术的互动和交流所定义。最近的大规模计算机评估开发了一些交互式问题解决项目和协作问题解决项目。调查人员将开发统计方法来分析这些新项目。研究人员将集中在几个在现代计算机评估分析中具有挑战性的方面；具体地说，他们将集中在：1)通过现代机器学习技术预测人类行为；2)通过事件历史分析从交互式问题解决项目收集的过程数据中提取潜在结构和图形结构；3)通过强化学习框架提供个性化学习材料；以及4)开发数值方法来随机或确定性地优化高维函数。将要开发的模型将结合潜在变量和图形方法以及高维数据的深度学习技术。对于建模过程数据，调查人员将采用自然语言处理的建模和分段技术方面的最新进展。在计算方面，研究人员将开发自适应Robbins-Monro随机逼近。优化算法将利用数值方法的最新进展来开发。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This research project will develop statistical methods for modern cognitive assessment. With the increasing use of computer-based testing, a variety of high-dimensional and complex structured data sets have been collected. The project will focus on statistical modeling and inference for large-scale data sets of complex structures. Specific topics to be addressed include the analysis of process data, adaptive learning through a reinforcement learning framework, and the development of computational methods for the models to be developed. The results of this research will provide a deeper understanding of the complex data structures collected in technology-rich interactive tasks. The project will shed light on items in learning and assessment environments that are delivered online both in client-server constellations and in stand-alone applications. The project will provide guidelines to improve item quality with a focus on more innovative item types, such as those in scenario-based and simulation-based environments for the assessment of students' knowledge and skills in the STEM fields. Educational researchers will be provided with tools to identify patterns in high-dimensional data and sequence data. Students in instructional and interventional programs will benefit from this research, especially in the STEM fields that are increasingly defined by digital media and technology-based interaction and communication.Recent large-scale computer-based assessments have developed a number of interactive problem-solving items and collaborative problem-solving items. The investigators will develop statistical methods for the analysis of these new items. The investigators will concentrate on several aspects that are very challenging in the analysis of modern computer-based assessment; specifically, they will focus on: 1) predicting human behavior by means of modern machine learning techniques; 2) extracting latent structure and graphical structure for process data collected by interactive problem-solving items through event history analyses; 3) providing personalized learning material through a reinforcement learning framework; and 4) developing numerical methods to optimize high-dimensional functions either stochastically or deterministically. The models to be developed will combine latent variable and graphical approaches as well as deep-learning techniques for high-dimensional data. For modeling process data, the investigators will employ recent advances in modeling and segmenting techniques for natural language processing. For computation, the investigators will develop adaptive Robbins-Monro stochastic approximation. Optimization algorithms will be developed using recent advances in numerical methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.