Modeling and Statistical Analysis of Mental Test Data

摘要本研究涉及多题（项）心理测试数据的概率建模与统计分析。研究者曾使用非参数项目反应理论（IRT）范式研究考试公平性、复杂潜在IRT结构的统计评估以及研究者共同开发的统一认知模型对应试考生的认知过程/心理测量建模和认知诊断。现在，有人提出了一种类似于经典的子测试分数水平因子分析法的非参数项目水平几何描述项目结构的方法。计划在统一模型的基础上开发一套实用的操作性认知测试程序。建议以各种急需的方式进一步发展测试偏置程序SIBTEST。跨越这三个领域的一个实际需求是对我们的非参数程序进行修改，以匹配基于潜在能力估计的可能性以及它们当前使用的数字正确分数。这项研究涉及开发数学模型和统计程序，以改进标准化和课堂测试，并改进对考生在这些测试中的表现的衡量。特别强调根据每个考生对个别考试问题的回答，为他/她提供具有教育意义的概念掌握概况，提供更公平的考试，并评估在考试中取得好成绩所需技能的实质性复杂性。特别是，希望(i)为每个项目提供它最能衡量的东西的描述，（ii）产生更公平的测试——特别是通过告知未来测试的结构规范，说明应该排除有偏见的项目类型，以及（iii）为教育工作者提供每个考生掌握和不掌握概念的详细反馈，用于指导进一步的指导和补救工作。由于标准化考试在美国社会中越来越普遍，并注意到其看门人的作用，上述工作如果成功，应该对美国标准化考试的有效性产生重大影响，并确实可以改善课堂上的教育过程。

Modeling and Statistical Analysis of Mental Test Data William Stout University of Illinois Abstract This research involves the probabilistic modeling and statistical analysis of multiple question(item) mental test data. The investigator has in the past used the nonparametric item response theory(IRT) paradigm to study test fairness, the statistical assessment of complex latent IRT structures, and the cognitive process/psychometric modeling and cognitive diagnosis of test taking examinees using the investigator co-developed Unified Cognitive Model. Now, it is proposed to develop a nonparametric item level geometric description of the item structure analogous to the classical subtest score level factor analytic approach. It is planned to develop a practical operational cognitive testing procedures based on the Unified model. It is proposed to further develop the test bias procedure, SIBTEST, in various much needed ways. A practical need cutting across all three areas is forour nonparametric procedures to be modified to match on liklihood based latent ability estimates as well as on their currently used number correct scores. The research involves developing mathematical models and statistical procedures to improve standardized and classroom tests and to improve the measurement of examinee performance on such tests. There is a special emphasis on providing an educationally useful concept mastery profile for each examinee based upon his/her individual test question responses, on providing fairer tests, and on assessing the substantive complexity of skills required for performing well on a test. In particular, it is desired to (i) provide for each item a description of what it measures best, (ii) to produce fairer tests -- in particular by informing the construction specifications of future tests about biased item types that should be excluded, and (iii) to provide for educators detail ed feedback of concept mastery and nonmastery for each individual examinee to be used to guide further instruction and remedial efforts. Because standardized tests are ever more ubiquitous in American society and noting their gatekeeper role, the above work, if successful, should have a major impact on how effective standardized tests in America are and indeed could improve the educational process in the classroom.