Semi-Parametric Factor Analysis for Item Responses and Response Times

这项研究项目将开发一个通用的建模框架，用于根据项目反应和反应时间数据来衡量潜在特征，如语言熟练程度和认知技能。响应时间已被确定为测试性能的有用指标。该项目将增加对响应时间在复杂的测试行为中的作用的理解，例如解决问题的策略和练习/疲劳的影响。对响应时间作用的研究将有助于测试/调查的开发和管理实践。将要开发的计量模型将依赖于很少的参数假设，这将使其适合于分析各种计算机化评估/在线调查数据。该项目将开发有效的算法来估计新的半参数模型。该项目将指导研究生，并开发开源软件以促进研究成果的传播。该项目将制定分类项目反应和连续反应时间的一般因素分析模型。为了获得最大的泛化能力，以潜在特征为条件的项目响应和响应时间的分布将用光滑的概率质量/密度函数来表征，并用带有方差参数化函数分析的张量积回归样条法来逼近。研究人员将开发和实现一种随机期望最大化算法，用于所提出的半参数模型的惩罚最大似然估计。为了避免过度拟合，他们将通过有效自由度和混合效果模型表示来检查平滑参数的经验选择。这些方法的性能将通过广泛的蒙特卡罗模拟进行评估。2015年国际学生评估计划的数据和关系推理测试-初级数据将被分析，以证明拟议方法的灵活性。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This research project will develop a generic modeling framework for measuring latent traits, such as language proficiency and cognitive skills, from item response and response-time data. Response time has been identified as a useful indicator for test performance. The project will increase understanding the role of response time in complex test-taking behaviors, such as problem-solving strategies and the practice/fatigue effect. Research on the role of response time will benefit the practice of test/survey development and administration. The measurement model to be developed will rely on very few parametric assumptions, which will make it appropriate for analyzing a wide variety of computerized assessment/online survey data. The project will develop efficient algorithms for estimating the new semi-parametric model. Graduate students will be mentored, and open-source software will be developed to facilitate the dissemination of the research product.This project will formulate a general factor analysis model for categorical item responses and continuous response times. To achieve maximal generalizability, the distributions of item responses and response times conditional on the latent traits will be characterized by smooth probability mass/density functions and approximated by tensor-product regression splines with a functional analysis of variance parameterization. The investigators will develop and implement a stochastic expectation-maximization algorithm for the penalized maximum likelihood estimation of the proposed semi-parametric model. To avoid overfitting, they will examine the empirical selection of smoothing parameters by effective degrees of freedom and a mixed-effects model representation. The performance of these methods will be assessed via extensive Monte Carlo simulations. The 2015 Programme for International Student Assessment data and the Test of Relational Reasoning-junior data will be analyzed to demonstrate the flexibility of the proposed method.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.