Semi-Parametric Factor Analysis for Item Responses and Response Times

本研究将开发一个通用的建模框架，用于测量潜在的特质，如语言能力和认知技能，从项目反应和反应时间数据。响应时间已被确定为测试性能的有用指标。该项目将增加对反应时间在复杂考试行为中的作用的理解，例如解决问题的策略和练习/疲劳效应。研究响应时间的作用将有助于测试/调查开发和管理的实践。将开发的计量模式将依赖于很少的参数假设，这将使其适合于分析各种各样的计算机化评估/在线调查数据。该项目将开发用于估计新的半参数模型的有效算法。将指导研究生，并将开发开放源码软件，以促进研究产品的传播，该项目将制定一个通用因素分析模型，用于分类项目反应和连续反应时间。为了实现最大的概括性，项目响应和响应时间的分布条件的潜在性状的特点是光滑的概率质量/密度函数和近似的张量产品回归样条函数的方差参数化的功能分析。研究人员将开发和实现一个随机期望最大化算法的惩罚最大似然估计的建议半参数模型。为了避免过度拟合，他们将通过有效自由度和混合效应模型表示来检查平滑参数的经验选择。这些方法的性能将通过广泛的蒙特卡罗模拟进行评估。2015年国际学生评估项目数据和关系推理测试-初级数据将被分析，以证明所提出的方法的灵活性。该奖项反映了NSF的法定使命，并已被认为是值得通过评估使用基金会的智力价值和更广泛的影响审查标准的支持。

This research project will develop a generic modeling framework for measuring latent traits, such as language proficiency and cognitive skills, from item response and response-time data. Response time has been identified as a useful indicator for test performance. The project will increase understanding the role of response time in complex test-taking behaviors, such as problem-solving strategies and the practice/fatigue effect. Research on the role of response time will benefit the practice of test/survey development and administration. The measurement model to be developed will rely on very few parametric assumptions, which will make it appropriate for analyzing a wide variety of computerized assessment/online survey data. The project will develop efficient algorithms for estimating the new semi-parametric model. Graduate students will be mentored, and open-source software will be developed to facilitate the dissemination of the research product.This project will formulate a general factor analysis model for categorical item responses and continuous response times. To achieve maximal generalizability, the distributions of item responses and response times conditional on the latent traits will be characterized by smooth probability mass/density functions and approximated by tensor-product regression splines with a functional analysis of variance parameterization. The investigators will develop and implement a stochastic expectation-maximization algorithm for the penalized maximum likelihood estimation of the proposed semi-parametric model. To avoid overfitting, they will examine the empirical selection of smoothing parameters by effective degrees of freedom and a mixed-effects model representation. The performance of these methods will be assessed via extensive Monte Carlo simulations. The 2015 Programme for International Student Assessment data and the Test of Relational Reasoning-junior data will be analyzed to demonstrate the flexibility of the proposed method.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.