Detecting Examinees With Pre-knowledge in Experimental Data Using Conditional Scaling of Response Times
Detecting Examinees With Pre-knowledge in Experimental Data Using Conditional Scaling of Response Times
复制标题
使用响应时间的条件缩放来检测具有实验数据预先知识的考生
DOI：
10.3389/feduc.2019.00049
复制
发表时间：
2019
影响因子：
2.3
通讯作者：
D. Maynes
中科院分区：
文献类型：
--
作者：
Sarah L. Toton;D. Maynes
在考试安全的背景下，检测以前访问过专有考试内容的考生是一个主要关注的问题。研究人员建议使用项目反应时间来检测考生的预知，但由于缺乏包含预知可信信息的真实数据和严格的统计假设，这一领域的进展受到限制。在这项工作中，提出了一种创新而简单的方法来检测具有预知识的考生。所提出的方法代表了一种条件缩放法，评估考生对特定问题的反应时间，与一组没有预知的考生相比，条件缩放法以问题是否回答正确为条件。以93名大学生为研究对象，随机分为有预知性组和无预知性组。参与者参加了一项计算机化的GRE定量推理测试，根据他们的情况，在测试前没有题目，一半题目，或者一半有正确答案的题目来学习。采用探索性分析技术对项目和个人水平的结果值进行调查，包括因素分析和聚类分析。所提出的方法在区分披露项目和未披露项目以及有无预先知识的考生方面取得了令人印象深刻的准确性（聚类分析的准确率分别为96%和97%），证明了对项目披露和考生预先知识的检测能力。该方法需要对数据的最小假设，可用于各种现代测试设计，排除其他类型的数据取证分析。
The detection of examinees who have previously accessed proprietary test content is a primary concern in the context of test security. Researchers have proposed using item response times to detect examinee pre-knowledge, but progress in this area has been limited by a lack of real data containing credible information about pre-knowledge and by strict statistical assumptions. In this work, an innovative, but simple, method is proposed for detecting examinees with pre-knowledge. The proposed method represents a conditional scaling that assesses an examinee’s response time to a particular item, compared to a group of examinees who did not have pre-knowledge, conditioned on whether or not the item was answered correctly. The proposed method was investigated in empirical data from 93 undergraduate students, who were randomly assigned to have pre-knowledge or not. Participants took a computerized GRE Quantitative Reasoning test and were given no items, half the items, or half the items with correct answers to study before the test, depending on their condition. Exploratory analysis techniques were used to investigate the resulting values at both the item and person-level, including factor analyses and cluster analyses. The proposed method achieved impressive accuracy of separation between disclosed and undisclosed items and examinees with and without pre-knowledge (96% and 97% accuracy for cluster analyses, respectively), demonstrating detection power for item disclosure and examinee pre-knowledge. The methodology requires minimal assumptions about the data and can be used for a variety of modern test designs that preclude other types of data forensic analyses.