Doctoral Dissertation Research: Multidimensional Nominal Response Models in Adaptive Testing

该奖项的全部或部分资金来自《2021年美国救援计划法案》(公法117-2)。本博士论文研究项目将探讨多维测验在计算机化适应性测验中的应用。在心理健康、教育能力和职业适宜性等问题上，多维测试被用于在许多学科中做出关键决定。其中一些测试是作为计算机化的适应性测试(CATS)进行的，这是一种根据考生的项目反应实时调整测试的测试模式。一般来说，CATS的设计是根据考生是否选择了正确的回答选项或错误的回答选项，来处理得分为0或1的项目反应。然而，这种二分法评分方法忽略了错误回答选项中包含的信息，这些错误回答选项可以被建模为名义回答数据。对单一受试者猫的研究表明，当其组件被设计为使用名义反应数据时，测试比为两种反应设计的测试更短、更准确。本项目将考察利用项目反应选项作为名义反应数据的多维自适应测验的价值。将开发心理测量软件并公开提供。作为博士论文研究改进奖，提供支持，使有前途的学生建立一个强大的，独立的研究生涯。这项研究项目将推动多维测试在计算机化适应性测试中的使用，通过将项目反应选项纳入构成适应性测试的建模、项目选择和评估过程中。该项目将对这种方法的准确性、效率和安全性与现有程序进行比较。多维自适应测试(MATS)和认知诊断-计算机化自适应测试(CD-CATS)的自适应测试组件和结果将被模拟。MATS和CD-CATS都可以用于终结性或形成性评估目的。然而，它们在理论假设上有所不同，因此必须单独进行调查。为了将多维适应性测试中项目反应选项的纳入与当前的二分法评分程序进行比较，将对题库和项目选择方法进行操作。为确保可比性，我们将推广常用的二分反应数据条目选择方法，以确保与名义反应数据的可比性。该奖项反映了NSF的法定使命，并已通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2). This doctoral dissertation research project will investigate the use of multidimensional tests in computerized adaptive testing. Multidimensional tests are used across many disciplines to make critical decisions on issues such as psychological health, educational ability, and vocational suitability. Some of these tests are administered as Computerized Adaptive Tests (CATs), a mode of testing that adapts the test in real time based on the examinee's item responses. Generally, CATs are designed to work with item responses that are scored as 0 or 1, based on whether the examinee has chosen the correct response option or an incorrect response option. However, this dichotomous scoring scheme disregards information contained in incorrect response options, which can be modeled as nominal response data. Research on single-subject CATs has demonstrated that when its components are designed to work with nominal response data, the tests were shorter and more accurate than those designed for dichotomous responses. This project will examine the value of multidimensional adaptive tests that utilize item response options as nominal response data. Psychometric software will be developed and made publicly available. As a Doctoral Dissertation Research Improvement award, support is provided to enable a promising student to establish a strong, independent research career.This research project will advance the use of multidimensional tests in computerized adaptive testing by incorporating item response options into the modeling, item selection, and estimation processes that constitute the adaptive test. The project will compare the accuracy, efficiency, and security of this approach to existing procedures. Adaptive test components and results will be simulated for Multidimensional Adaptive Tests (MATs) and Cognitive Diagnostic-Computerized Adaptive Tests (CD-CATs). Both MATs and CD-CATs can be used for either summative or formative assessment purposes. However, they differ in their theoretical assumptions and thus must be investigated separately. To compare the incorporation of item response options in multidimensional adaptive testing to current dichotomous scoring procedures, item banks and item selection methods will be manipulated. Common item selection methods for dichotomous response data will be generalized for use with nominal response data to ensure comparability.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.