Developing techniques for constructing and maintaining an item pool

在严密的协调下，由生物统计学家、心理测量学家和医学教育工作者组成的三个小组进行了本研究。本研究的主要目的是开发用于构建和维护题库的量化技术。特别努力改进正在进行的共同成就测试的计算机测试(CBT)。通过与实际项目的密切接触，使本研究具有一定的实用性。这一CBT在大量正在进行的测试中确实是一个突破。它是世界上最具挑战性的教育测试之一。测试结果根据标准项目反应理论进行分析。这一分析有助于制定试验项目的纳入-排除标准。此外，还提出了需要研究的各种重要课题。其中之一是证明为什么题库对于充分评估学生特定学术领域的知识是必要的。这允许我们ri…更科学的教育考试基础，更灵活的全国考试操作。另一方面，试题效度、试题数量庞大、试题特征量化评价等问题也随之而来。另一个重要的问题是如何在题库中获得可靠的定量项目参数。除了分析技术外，我们还强调了基于项目反应理论的精心设计的预操作的必要性。这一必要性原则上得到了研究人员的广泛接受，但实际实施和实际使用都没有尝试过。幸运的是，共同成就测试委员会接受了我们的建议。这可能是大规模测试中的第一次试验。该结果将有助于评估项目参数，特别是在怀疑总体时间趋势的情况下。通过贝叶斯分析，抽样技术正在迅速发展。预计它们将有助于平衡为特定考生选择的试题的各个方面。对考生的意见和印象进行了一次小型调查。这一结果提醒我们注意物品中仍有意想不到的不清楚描述。更重要的是，他们建议考生如何作为重要的利益相关者参与CBT。这种观点在现有的测试中被完全抛弃了。除了上述项目外，我们还参与了其他重要项目。其中之一是通过共同成就测试的欧安组织。其他包括LSAT和针对护生的CBT。这些项目使本研究在实践中有所帮助，我们可以期待对未来的教育测试做出可能的贡献。较少

Under tight coordination the three groups, consisting of biostatisticians, psychometricians and medical educators, conducted the present research. The primary aim of the present research is to develop quantitative techniques for constructing and maintaining an item pool. Special efforts are placed on improving the ongoing the computer based testing (CBT) of the Common Achievement Test. Intensive contacts with this actual project make this research practical. This CBT is really a breakthrough in a huge number of ongoing testings. It is one of most challenging educational tests in the world.The result of trials are analyzed in terms of the standard item response theory. This analysis is helpful for preparing inclusion-exclusion criteria of items tested in the trials. Further it suggests various important subjects to be studied. One of them is to justify why an item pool is necessary for evaluating adequately knowledges regarding a specified academic field of an student. This allows us ri … More gid scientific bases of educational testings and also flexible operation of nation-wide testing. On the other hand, various difficulties give rise to, which include validity of items, a huge number of items in a pool, quantitative evaluation of item characteristics. Another important problem is how we obtain reliable quantitative item parameters in a item pool. In addition to analytical techniques we emphasized the need for a well designed pre-operation based on the item response theory. The necessity is in principle widely accepted among researchers but practical implementation, together with its actual use, has not been attempted. Fortunately, the Board of the Common Achievement Test accepted our propose. Probably this is the first trial in large-scaled testings. The result will be useful for evaluating item parameters, especially when the general time trend will be suspected. Sampling techniques are rapidly developing though Bayesian analysis. They are expected to be useful in balancing various aspects of items selected to a particular examinee. A small size survey regarding opinions and impressions of examinees was conducted. The results call our attention to unexpected unclear descriptions remained in items. More importantly, they suggest how examinees participate the CBT as an important stakeholder. Such a view was completely discarded in the existing testings.In addition to the above project, we participate other important projects. One of them is OSCE by the Common Achievement Testing. The others include LSAT and the CBT for nurse students. These projects make the present research useful in practice, and we can expect possible contributions on future educational tests. Less