Advances in Computerized Adaptive Testing: Modeling Response Times and Constraint Management for Skills Diagnosis

计算机化自适应测试的进展：技能诊断的响应时间建模和约束管理

• 批准号: 0960822
• 负责人: Jeffrey Douglas
• 金额: $ 23.31万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-06-01 至 2012-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0960822&HistoricalAwards=false
• 关键词: Advances; Computerized; Adaptive; Testing; Modeling

Computerized Adaptive Testing (CAT) has become popular in high-stakes testing programs. Examples of large-scale CATs include the Graduate Record Examination, the Graduate Management Admission Test, the National Council of State Boards Nursing, and the Armed Services Vocational Aptitude Battery. An advantage of CAT over paper-and-pencil exams is that it provides more efficient estimation of abilities because it can appropriately tailor item selection to the estimated abilities of examinees. This research addresses two areas of critical importance for CAT. First, flexible models for response times will be developed to assist in controlling the duration of an exam, and also to assist in the measurement of ability in appropriate circumstances. Second, statistical methods for constraint management will be developed to ensure that an exam has sufficient information to diagnose fine-grained skills while also providing an accurate summary score.The impact of the research will be to provide technology to better utilize response-time information and also enhance the ability of CAT to provide diagnostic information. Models for response-time distributions will be developed that make few assumptions concerning functional form and allow for dependence between response times and a latent trait that represents ability on the studied domain. Estimation techniques will be developed that may be used with data previously collected from exams administered with CAT. Algorithms for utilizing estimated response-time distributions will be constructed to better manage duration of exams and to extract information from response times to better estimate the ability the exam is designed to measure. In addition to addressing response times, the problem of managing the diagnostic information to assess mastery of fine-grained skills will be studied for exams that also aim to provide a single summary score. CAT methods for adaptively selecting items to more efficiently provide a score will be modified to simultaneously balance the coverage of skills and attributes of interest.

计算机化的自适应测试(CAT)在高风险的测试程序中已经变得流行起来。大规模CATS的例子包括研究生入学考试、研究生管理入学考试、国家委员会护士委员会和军队职业能力小组。与纸笔考试相比，CAT的一个优点是它提供了更有效的能力估计，因为它可以根据考生估计的能力适当地调整项目选择。这项研究涉及对CAT至关重要的两个领域。首先，将开发灵活的响应时间模型，以帮助控制考试的持续时间，并帮助在适当的情况下测量能力。其次，将开发约束管理的统计方法，以确保考试有足够的信息来诊断细粒度的技能，同时提供准确的总结分数。研究的影响将是提供技术，以更好地利用响应时间信息，并增强CAT提供诊断信息的能力。将开发响应时间分布模型，该模型几乎不对函数形式做出假设，并允许响应时间与代表所研究领域能力的潜在特征之间的相关性。将开发评估技术，可与先前从CAT管理的检查中收集的数据一起使用。将构建利用估计响应时间分布的算法，以更好地管理考试持续时间，并从响应时间中提取信息，以更好地估计考试旨在测量的能力。除了解决响应时间问题外，还将研究如何管理诊断信息以评估对细粒度技能的掌握情况，以应对也旨在提供单一汇总分数的考试。用于自适应地选择项目以更有效地提供分数的CAT方法将被修改，以同时平衡技能和感兴趣的属性的复盖面。