Modeling and Statistical Analysis of Mental Test Data

心理测验数据的建模和统计分析本研究涉及对多问题(项目)心理测验数据的概率建模和统计分析。研究者过去曾使用非参数项目反应理论(IRT)范式来研究考试公平性、复杂潜在IRT结构的统计评估，以及使用研究者共同开发的统一认知模型对应试者的认知过程/心理测量模型和认知诊断进行研究。现在，有人提出了一种类似于经典的分测验分数水平因素分析方法的非参数项目水平几何描述。计划在统一模型的基础上开发实用的可操作性认知测试程序。建议以各种急需的方式进一步发展测试偏置程序SIBTEST。一个跨越所有三个领域的实际需要是修改我们的非参数程序，以匹配基于似然度的潜在能力估计以及他们当前使用的数字正确分数。这项研究涉及开发数学模型和统计程序，以改进标准化和课堂测试，并改进对考生在此类测试中的表现的测量。它特别强调根据每个考生的个人考试问题回答为他/她提供一个在教育上有用的概念掌握概况，提供更公平的考试，并评估在考试中表现良好所需技能的实质复杂性。具体而言，我们希望(I)为每道试题提供其衡量最佳内容的描述，(Ii)制定更公平的考试--特别是通过告知未来考试的结构规范，说明应排除的有偏见的试题类型，以及(Iii)为教育工作者提供关于每个考生对概念掌握情况和不掌握情况的详细反馈，以用于指导进一步的指导和补救工作。由于标准化考试在美国社会中越来越普遍，并注意到它们的守门人角色，上述工作如果成功，应该会对美国标准化考试的有效性产生重大影响，甚至可以改善课堂教育过程。

Modeling and Statistical Analysis of Mental Test Data William Stout University of Illinois Abstract This research involves the probabilistic modeling and statistical analysis of multiple question(item) mental test data. The investigator has in the past used the nonparametric item response theory(IRT) paradigm to study test fairness, the statistical assessment of complex latent IRT structures, and the cognitive process/psychometric modeling and cognitive diagnosis of test taking examinees using the investigator co-developed Unified Cognitive Model. Now, it is proposed to develop a nonparametric item level geometric description of the item structure analogous to the classical subtest score level factor analytic approach. It is planned to develop a practical operational cognitive testing procedures based on the Unified model. It is proposed to further develop the test bias procedure, SIBTEST, in various much needed ways. A practical need cutting across all three areas is forour nonparametric procedures to be modified to match on liklihood based latent ability estimates as well as on their currently used number correct scores. The research involves developing mathematical models and statistical procedures to improve standardized and classroom tests and to improve the measurement of examinee performance on such tests. There is a special emphasis on providing an educationally useful concept mastery profile for each examinee based upon his/her individual test question responses, on providing fairer tests, and on assessing the substantive complexity of skills required for performing well on a test. In particular, it is desired to (i) provide for each item a description of what it measures best, (ii) to produce fairer tests -- in particular by informing the construction specifications of future tests about biased item types that should be excluded, and (iii) to provide for educators detail ed feedback of concept mastery and nonmastery for each individual examinee to be used to guide further instruction and remedial efforts. Because standardized tests are ever more ubiquitous in American society and noting their gatekeeper role, the above work, if successful, should have a major impact on how effective standardized tests in America are and indeed could improve the educational process in the classroom.