A 'compensatory selection' effect with standardized tests: Lack of correlation between test scores and success is evidence that test scores are predictive of success.
A 'compensatory selection' effect with standardized tests: Lack of correlation between test scores and success is evidence that test scores are predictive of success.
复制标题
DOI：
10.1371/journal.pone.0265459
复制
发表时间：
2022
期刊：
影响因子：
3.7
通讯作者：
Staub, Adrian
中科院分区：
文献类型：
--
作者：
Huber, David E.;Cohen, Andrew L.;Staub, Adrian
我们引入了“补偿选择”的统计概念，这是在基于多个预测因素选择申请人子集时产生的，例如当标准化考试成绩与学校申请中所需的其他预测因素结合使用时（例如，以前的成绩、推荐信和个人陈述）。事后分析往往无法找到考试成绩和随后的成功之间的正相关关系，这种失败有时被视为对标准化考试的预测有效性的证据。目前的分析表明，未能找到一个负相关表明，标准化考试实际上是一个有效的预测成功。这是由于在选择过程中预测因素之间的补偿：一些学生尽管考试成绩较低，但仍被录取，因为他们的申请在其他方面很出色，而其他学生则主要基于考试成绩高而被录取，尽管他们的申请在其他方面很薄弱。这种补偿性的选择过程在被录取的学生中引入了考试成绩和其他预测因素之间的负相关（“碰撞偏差”或“Berkson悖论”效应）。如果考试成绩是成功的有效预测因素，那么预测因素之间的负相关性就会抵消考试成绩与成功之间的正相关性，如果所有申请人都被录取，就会观察到这种正相关性。如果测试分数不能预测成功，但仍然用于补偿选择过程，则测试分数和成功之间将存在虚假的负相关（即，一个被录取的学生，除了考试分数高之外，申请能力差，不太可能成功）。这里所描述的选择效应与众所周知的“限制范围”问题有着根本的不同，即使在接受大多数申请人的情况下，也会有力地改变结果。
We introduce the statistical concept of ’compensatory selection’, which arises when selecting a subset of applicants based on multiple predictors, such as when standardized test scores are used in combination with other predictors required in a school application (e.g., previous grades, references letters, and personal statements). Post-hoc analyses often fail to find a positive correlation between test scores and subsequent success, and this failure is sometimes taken as evidence against the predictive validity of the standardized test. The present analysis reveals that the failure to find a negative correlation indicates that the standardized test is in fact a valid predictor of success. This is due to compensation between predictors during selection: Some students are admitted despite a low test score because their application is exceptional in other respects, while other students are admitted primarily based on a high test score despite weakness in the rest of their application. This compensatory selection process introduces a negative correlation between test scores and other predictors among those admitted (a ’collider bias’ or ’Berkson’s paradox’ effect). If test scores are valid predictors of success, this negative correlation between the predictors counteracts the positive correlation between test scores and success that would have been observed if all applicants were admitted. If test scores are not predictive of success, but were nevertheless used in a compensatory selection process, there would be a spurious negative correlation between test scores and success (i.e., an admitted student with a weak application except for a high test score would be unlikely to succeed). The selection effect that is described here is fundamentally different from the well-known ’restricted range’ problem and can powerfully alter results even in situations that accept most applicants.
登录
查看更多内容
影响因子：
22.4
作者：
Kuncel, NR;Hezlett, SA;Ones, DS
通讯作者：
Ones, DS
影响因子：
3.7
作者：
Moneta-Koehler, Liane;Brown, Abigail M.;Chalkley, Roger
通讯作者：
Chalkley, Roger
影响因子：
13.6
作者：
Miller, Casey W.;Zwickl, Benjamin M.;Hodapp, Theodore
通讯作者：
Hodapp, Theodore
影响因子：
3.7
作者：
Petersen, Sandra L.;Erenrich, Evelyn S.;Gile, Krista
通讯作者：
Gile, Krista