Accuracy of the Q-matrix-validation algorithm and sensitivity of model fit indices toward different types of Q-matrix misspecification for diagnostic classification models (DiaFit)

Q 矩阵验证算法的准确性和模型拟合指数对诊断分类模型 (DiaFit) 不同类型 Q 矩阵错误指定的敏感性

• 批准号: 520899465
• 负责人: Professorin Dr. Olga Kunina-Habenicht, Ph.D.
• 金额: --
• 依托单位: Fachgebiet Rehabilitationspsychologie
• 依托单位国家: 德国
• 项目类别: Research Grants
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/520899465?language=en
• 关键词: Accuracy; matrix; validation; algorithm; sensitivity

Diagnostic classification models (DCM) represent an internationally widely discussed but in Germany little known class of confirmatory probabilistic multidimensional latent-variable models with categorical latent variables providing meaningful proficiency profiles based on statistically-driven multivariate classifications. These profiles can inform students and teachers about the particular strengths and weaknesses in a given academic area and allow for the derivation of appropriate supportive interventions. For the estimation of DCM the specification of attributes involved in the solution process of items in a Q-matrix is required. The determination of the Q-matrix is of key importance, as it represents the theoretical basis of DCM. In practical applications, however, the true Q-matrix is unknown and may contain incorrect entries. This phenomenon is called Q-matrix misspecification. It is common to distinguish between three different types of Q-matrix misspecification: underspecified, overspecified, and balanced. Several algorithms have been proposed in the literature for the empirical validation of the Q-matrix. However, their accuracy has been investigated only for selected types of Q-matrix misspecification. Moreover, assessing model fit for DCM continues to be a challenge, although various absolute and relative model fit indices for DCM have been proposed in the past. The project DiaFit investigates in a complex simulation study and an application study important methodological issues related to the assessment of the model fit and the accuracy of the Q-matrix validation algorithm for different types of Q-matrix misspecification for log-linear DCM. In the simulation study the sensitivity of different absolute and relative model fit measures (e.g. AIC, BIC, MAD, RMSEA) to three different types of misspecified Q-matrices (underspecified, overspecified, balanced) will be investigated. Moreover, the accuracy of the Q-matrix validation algorithm for these three types of Q-matrix misspecification will be addressed. In the simulation the number of respondents, attributes, and items as well as the item-structure complexity in the Q-matrix will be varied. The results will essentially extend the state of the art in model fit assessment and robustness of the Q-matrix validation algorithm for log-linear DCM and enable recommendations for the assessment of the model fit of DCM in applied settings. The results of the simulation will be applied in a secondary analysis to two different datasets on basic arithmetic skills in elementary school that have already been collected. The Q-matrices are also available for these datasets. This application study will illustrate how different model fit indices and the Q-matrix validation method can be used in practical applications when the true data-generating Q-matrix is unknown.

诊断分类模型(DCM)代表了一类在国际上被广泛讨论但在德国鲜为人知的验证性概率多维潜变量模型，它的分类潜变量基于统计驱动的多变量分类提供有意义的熟练程度轮廓。这些简介可以让学生和教师了解特定学术领域的特殊优势和劣势，并允许得出适当的支持性干预措施。对于DCM的估计，需要对Q矩阵中的项的求解过程中涉及的属性进行说明。Q矩阵的确定至关重要，因为它代表了DCM的理论基础。然而，在实际应用中，真实的Q矩阵是未知的，并且可能包含不正确的条目。这种现象被称为Q-矩阵误指定。常见的是区分三种不同类型的Q矩阵错误指定：指定不足、指定过多和平衡。在文献中已经提出了几种用于Q矩阵的经验验证的算法。然而，它们的准确性仅在选定类型的Q矩阵错误指定时进行了研究。此外，尽管过去已经提出了各种用于DCM的绝对和相对模型拟合指数，但评估DCM的模型适配性仍然是一个挑战。DiaFit项目在一项复杂的模拟研究和一项应用研究中研究了重要的方法学问题，这些问题涉及对数线性DCM的不同类型的Q矩阵错误指定的Q矩阵验证算法的模型拟合和精度评估。在模拟研究中，将研究不同的绝对和相对模型拟合度(例如AIC、BIC、MAD、RMSEA)对三种不同类型的错误指定Q-矩阵(不足指定、过度指定、平衡)的敏感度。此外，对于这三种类型的Q矩阵错误说明，Q矩阵验证算法的准确性也将得到解决。在模拟中，Q矩阵中的受访者数量、属性和条目以及条目结构的复杂性将是不同的。该结果将从本质上扩展对数线性DCM的Q-矩阵验证算法在模型拟合评估和稳健性方面的最新水平，并能够为在应用环境中评估DCM的模型拟合提供建议。模拟的结果将被应用于对已经收集的两个不同的小学基本算术技能数据集的二次分析。Q矩阵也可用于这些数据集。这项应用研究将说明在真实数据生成Q-矩阵未知的情况下，如何在实际应用中使用不同的模型拟合指数和Q-矩阵验证方法。