Developing techniques for constructing and maintaining an item pool

在紧密的协调下，由生物统计学家、心理测量学家和医学教育家组成的三个小组进行了本研究。本研究的主要目的是开发定量技术，用于构建和维护一个项目库。我们特别努力改进正在进行的基于计算机的通用成绩测试（CBT）。与实际项目的紧密联系使本研究具有实用性。这个CBT确实是大量正在进行的测试中的一个突破。它是世界上最具挑战性的教育测验之一，本文根据标准项目反应理论对测试结果进行了分析。该分析有助于制定试验中检测项目的纳入-排除标准。此外，它提出了各种重要的课题要研究。其中之一是证明为什么一个项目池是必要的，充分评估知识，关于一个学生的特定学术领域。这使我们能够 ...更多信息 教育考试的科学性和全国性考试的灵活性。另一方面，也带来了各种困难，包括项目的有效性、项目库中的大量项目、项目特征的定量评价等。另一个重要的问题是我们如何获得可靠的定量项目参数在一个项目池。除了分析技术，我们强调需要一个精心设计的前操作的项目反应理论的基础上。研究人员原则上广泛接受这种必要性，但尚未尝试实际实施及其实际使用。幸运的是，共同成就测试委员会接受了我们的建议。这可能是大规模试验的第一次尝试。该结果将有助于评估项目参数，特别是当一般的时间趋势将被怀疑。通过贝叶斯分析，抽样技术得到了迅速发展。他们预计将是有用的，在平衡的各个方面选择一个特定的考生的项目。对考生的意见和印象进行了小规模的调查。这些结果提醒我们注意项目中存在的意想不到的不明确描述。更重要的是，他们建议考生如何参与CBT作为一个重要的利益相关者。这种观点在现有的测试中被完全抛弃。除了上述项目外，我们还参与了其他重要项目。其中之一是欧安组织的共同成就测试。其他考试包括LSAT和护士学生的CBT。这些项目使目前的研究在实践中有用，我们可以期待未来的教育测试可能的贡献。少

Under tight coordination the three groups, consisting of biostatisticians, psychometricians and medical educators, conducted the present research. The primary aim of the present research is to develop quantitative techniques for constructing and maintaining an item pool. Special efforts are placed on improving the ongoing the computer based testing (CBT) of the Common Achievement Test. Intensive contacts with this actual project make this research practical. This CBT is really a breakthrough in a huge number of ongoing testings. It is one of most challenging educational tests in the world.The result of trials are analyzed in terms of the standard item response theory. This analysis is helpful for preparing inclusion-exclusion criteria of items tested in the trials. Further it suggests various important subjects to be studied. One of them is to justify why an item pool is necessary for evaluating adequately knowledges regarding a specified academic field of an student. This allows us ri … More gid scientific bases of educational testings and also flexible operation of nation-wide testing. On the other hand, various difficulties give rise to, which include validity of items, a huge number of items in a pool, quantitative evaluation of item characteristics. Another important problem is how we obtain reliable quantitative item parameters in a item pool. In addition to analytical techniques we emphasized the need for a well designed pre-operation based on the item response theory. The necessity is in principle widely accepted among researchers but practical implementation, together with its actual use, has not been attempted. Fortunately, the Board of the Common Achievement Test accepted our propose. Probably this is the first trial in large-scaled testings. The result will be useful for evaluating item parameters, especially when the general time trend will be suspected. Sampling techniques are rapidly developing though Bayesian analysis. They are expected to be useful in balancing various aspects of items selected to a particular examinee. A small size survey regarding opinions and impressions of examinees was conducted. The results call our attention to unexpected unclear descriptions remained in items. More importantly, they suggest how examinees participate the CBT as an important stakeholder. Such a view was completely discarded in the existing testings.In addition to the above project, we participate other important projects. One of them is OSCE by the Common Achievement Testing. The others include LSAT and the CBT for nurse students. These projects make the present research useful in practice, and we can expect possible contributions on future educational tests. Less