Doctoral Dissertation Research: Multidimensional Nominal Response Models in Adaptive Testing

该奖项全部或部分由《2021年美国救援计划法案》（公法117-2）资助。本博士论文研究项目将探讨多维测试在计算机化自适应测试中的应用。多维测试在许多学科中被用于对心理健康、教育能力和职业适用性等问题做出关键决策。其中一些测试被称为计算机适应性测试（CATs），这是一种基于考生的项目反应实时调整测试的测试模式。一般来说，cat的设计是根据考生选择的是正确的回答选项还是错误的回答选项来处理得分为0或1的项目回答。然而，这种二分计分方案忽略了错误回答选项中包含的信息，这些信息可以建模为名义回答数据。对单主题cat的研究表明，当其组成部分被设计用于标称反应数据时，测试比为二分反应设计的测试更短，更准确。本项目将检验利用项目反应选项作为标称反应数据的多维自适应测试的价值。心理测量软件将被开发并向公众开放。作为博士论文研究改进奖，提供支持，使有前途的学生建立一个强大的，独立的研究生涯。本研究项目将通过将项目反应选项纳入构成自适应测试的建模、项目选择和估计过程，促进多维测试在计算机化自适应测试中的应用。该项目将比较这种方法与现有程序的准确性、效率和安全性。将模拟多维适应测试（MATs）和认知诊断-计算机化适应测试（CD-CATs）的适应性测试组件和结果。MATs和CD-CATs都可以用于总结性或形成性评估目的。然而，它们的理论假设不同，因此必须分开研究。为了比较多维自适应测试中项目反应选项与当前二分计分程序的结合，将对题库和项目选择方法进行操作。二分类反应数据的常见项目选择方法将推广到标称反应数据中，以确保可比性。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2). This doctoral dissertation research project will investigate the use of multidimensional tests in computerized adaptive testing. Multidimensional tests are used across many disciplines to make critical decisions on issues such as psychological health, educational ability, and vocational suitability. Some of these tests are administered as Computerized Adaptive Tests (CATs), a mode of testing that adapts the test in real time based on the examinee's item responses. Generally, CATs are designed to work with item responses that are scored as 0 or 1, based on whether the examinee has chosen the correct response option or an incorrect response option. However, this dichotomous scoring scheme disregards information contained in incorrect response options, which can be modeled as nominal response data. Research on single-subject CATs has demonstrated that when its components are designed to work with nominal response data, the tests were shorter and more accurate than those designed for dichotomous responses. This project will examine the value of multidimensional adaptive tests that utilize item response options as nominal response data. Psychometric software will be developed and made publicly available. As a Doctoral Dissertation Research Improvement award, support is provided to enable a promising student to establish a strong, independent research career.This research project will advance the use of multidimensional tests in computerized adaptive testing by incorporating item response options into the modeling, item selection, and estimation processes that constitute the adaptive test. The project will compare the accuracy, efficiency, and security of this approach to existing procedures. Adaptive test components and results will be simulated for Multidimensional Adaptive Tests (MATs) and Cognitive Diagnostic-Computerized Adaptive Tests (CD-CATs). Both MATs and CD-CATs can be used for either summative or formative assessment purposes. However, they differ in their theoretical assumptions and thus must be investigated separately. To compare the incorporation of item response options in multidimensional adaptive testing to current dichotomous scoring procedures, item banks and item selection methods will be manipulated. Common item selection methods for dichotomous response data will be generalized for use with nominal response data to ensure comparability.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.