Developing techniques for constructing and maintaining an item pool

点击翻译按钮获取中文摘要

Under tight coordination the three groups, consisting of biostatisticians, psychometricians and medical educators, conducted the present research. The primary aim of the present research is to develop quantitative techniques for constructing and maintaining an item pool. Special efforts are placed on improving the ongoing the computer based testing (CBT) of the Common Achievement Test. Intensive contacts with this actual project make this research practical. This CBT is really a breakthrough in a huge number of ongoing testings. It is one of most challenging educational tests in the world.The result of trials are analyzed in terms of the standard item response theory. This analysis is helpful for preparing inclusion-exclusion criteria of items tested in the trials. Further it suggests various important subjects to be studied. One of them is to justify why an item pool is necessary for evaluating adequately knowledges regarding a specified academic field of an student. This allows us ri … More gid scientific bases of educational testings and also flexible operation of nation-wide testing. On the other hand, various difficulties give rise to, which include validity of items, a huge number of items in a pool, quantitative evaluation of item characteristics. Another important problem is how we obtain reliable quantitative item parameters in a item pool. In addition to analytical techniques we emphasized the need for a well designed pre-operation based on the item response theory. The necessity is in principle widely accepted among researchers but practical implementation, together with its actual use, has not been attempted. Fortunately, the Board of the Common Achievement Test accepted our propose. Probably this is the first trial in large-scaled testings. The result will be useful for evaluating item parameters, especially when the general time trend will be suspected. Sampling techniques are rapidly developing though Bayesian analysis. They are expected to be useful in balancing various aspects of items selected to a particular examinee. A small size survey regarding opinions and impressions of examinees was conducted. The results call our attention to unexpected unclear descriptions remained in items. More importantly, they suggest how examinees participate the CBT as an important stakeholder. Such a view was completely discarded in the existing testings.In addition to the above project, we participate other important projects. One of them is OSCE by the Common Achievement Testing. The others include LSAT and the CBT for nurse students. These projects make the present research useful in practice, and we can expect possible contributions on future educational tests. Less