Qualitative and Quantitative Analyses of English Language Test Items in Japanese University Entrance Examinations

作为整个研究项目的前期准备，开发了测试项目规范的分析框架。该框架从任务类型和反应类型两个主要方面规定了项目的特征，并详细说明了项目的其他特征，如惯用语和被测技能(S)。此外，对于多项选择题，从选择项和选择项的数量(和顺序)方面进一步分析了回答类型。在开发这一框架的过程中，我们从随机选择的109所日本大学的入学考试和国家中心考试的三个版本中收集了2944个英语试题。国家中心考试是由国家大学入学考试中心开发和管理的一项成就测试。这些测试项目被用来确定我们框架的适用性。由于我们发现了比预期更广泛的任务类型和响应类型，因此进行了必要的修改，以使框架成为应用程序…在日本的EFL语境中更容易理解。利用这个修订后的框架，对同一组试题进行了分析，得到了日本大学入学考试英语试题的原型数据库。这个数据库揭示了一些大体趋势。例如，尽管最近转向了更具交际性的语言教学，但“技能和成分”的语言测试模式显然仍然在大学入学考试中占据主导地位。此外，尽管希望利用更广泛的可用的任务类型，但传统的类型仍然被用于测试读写技能。本研究利用国家中心考试题库，在提供了充分的统计信息的基础上，阐明了(I)具有独特统计趋势的题项的质量特征，并通过IRT对不同版本的国家中心考试进行等值，(Ii)日本英语学习者在大学入学时的学业成就变化。较少

As the preliminary to the whole research project, an analytical framework was developed for test item specification. This framework specifies characteristics of an item in two main aspects, i.e. task types and response types, and also details its other features such as rubrics and the skill(s) being tested. Furthermore, for the multiple-choice questions, the response type is further analyzed in terms of choices and the number (and order) of choices to an item. In developing this framework, we collected 2,944 English test items from entrance examinations of 109 randomly-selected Japanese universities and also from three versions of the National Center Test, an achievement test developed and administered by the National Center for University Entrance Examinations. These test items were used to determine the applicability of our framework. Since we found a wider than expected variety of task types and response types, necessary modifications were made so that the framework would be applica … More ble in the Japanese EFL context. Employing this revised framework, the same set of items were analyzed, yielding a prototypical database of English test items in university entrance examinations in Japan. This database reveals some general tendencies. For instance, despite a recent shift to more communicative language teaching, a "skills and components" model of language tests is clearly still dominant in university entrance examinations. In addition, although it would be desirable to make use of the wider variety of task types which are available, conventional varieties are still being used in the testing of reading and writing skills.Simultaneously, another test item database, viz. the database of the National Center Test items with statistical information fully provided, was employed to elucidate (i) qualitative characteristics of items which displayed unique statistical tendencies, and, by IRT-based equating of different versions of the National Center Test, (ii) changes in academic achievements of Japanese EFL learners at the time of university admission. Less