Understanding How Approaches to Calibrating and Scoring Survey Item Responses Affect Results from Growth Mixture Models

研究人员、教育工作者、心理学家和临床医生经常想知道个体是如何成长和发展的。此外，人们感兴趣的是，根据他们的发展，是否存在不同的——通常是看不见的——儿童或成人群体。例如，研究人员可能想要了解一个特定孩子的自我控制发展是否典型，或者在学生群体学习数学的过程中是否存在独特的模式。增长混合模型（gmm）是为这个目的而设计的统计工具：识别不同的增长模式，包括遵循这些模式的个体群体，当这些群体不可见时。尽管近几十年来gmm在发展研究中得到了广泛的应用，但这些工具的最佳实践尚未完全确立。特别是，一个重要的问题仍然没有得到解决：当长期管理的自我报告测量的分数中存在常见形式的测量偏差时（例如，在整个中学期间对学生进行的自我控制调查），使用这些分数的GMM结果的可信度如何？其中，GMM中一个突出的问题是他们是如何受到回答风格偏差的影响的，这种偏差可能发生在被调查者有回答问题的特征模式时，而不管问题的内容是什么，比如总是选择最高或最低的回答选项。未能应用解释反应风格的评分模型可能会产生有偏差的分数估计-但这对基于这些分数的GMM结果的影响程度是未知的。当前的项目使用项目反应理论（IRT）的进展来检查评分对GMM中的类别恢复和参数估计的影响——包括未能具体说明反应风格，以及更广泛地说明评分模型的错误。这是通过蒙特卡罗模拟和分析经验数据来实现的。在模拟研究中，生成具有不同特征的人工数据——包括样本量、GMM中的类数以及不同的测量问题（如反应风格偏差）——允许全面检查评分模型在何时以及在何种条件下对GMM产生错误规范。该实证研究将GMMs应用于两项大型研究的社会情感发展分数，从而深入了解发展科学中评分决策的现实后果。最后，将根据研究结果以“测量清单”的形式为研究人员提供指导。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Researchers, educators, psychologists, and medical clinicians often want to know how individuals grow and develop. Further, there is interest in whether there are distinct--and oftentimes unseen--groups of children or adults based on their development. For example, researchers might want to understand whether a given child’s development of self-control is typical or not, or whether there are distinct patterns in how groups of students learn math as they move through school. Growth mixture models (GMMs) are statistical tools designed for exactly that purpose: identifying distinct growth patterns, including groups of individuals who follow those patterns, when such groupings remain unseen. Although GMMs have seen widespread use in developmental research in recent decades, best practices for such tools are not fully established. In particular, one important question remains unaddressed: when common forms of measurement bias are present in scores from self-report measures administered over time (e.g., self- control surveys given to students throughout middle school), how trustworthy are GMM results using these scores? Among others, one outstanding issue in GMM is how they are affected by response style bias, which may occur when respondents have characteristic patterns of responding to questions regardless of question content, such as always picking the highest or lowest response option. Failure to apply scoring models which account for response styles may yield biased score estimates--but the extent to which this impacts GMM results based on these scores is unknown.The current project uses advances in item response theory (IRT) to examine the effects of scoring--including failure to account for response styles specifically, as well as mis-specification of the scoring model more broadly--on class recovery and parameter estimates in GMM. This is accomplished both through Monte Carlo simulation and analysis of empirical data. In the simulation study, the generation of artificial data with different features--including sample size, number of classes in the GMM, and different measurement issues such as response style bias--allows a comprehensive examination of when and under what conditions scoring model mis-specification matters for GMMs. The empirical study, which applies GMMs to scores of socioemotional development from two large studies, permits insight into the real-world consequences of scoring decisions in developmental science. Finally, guidance for researchers will be made available in the form of a ‘measurement checklist’ informed by the study results.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.