Computerized Adaptive Testing for Survey Research

该项目将推进计算机化自适应测试(CAT)，作为防静电电池的替代方法。社会科学家了解个人特征和信仰的一种常见方式是让人们回答许多关于同一主题的密切相关的问题。这些大型多项电池被用来测量人格特征、心理疾病、政治意识形态、个人价值观、知识水平等结构。然而，民意研究人员出于对受访者负担或长期调查成本的担忧，经常选择不使用这些广受推崇的多项电池。标准的解决方案是选择可用项目的子集，然后将其管理给所有受访者。然而，众所周知，这种减少静电的电池会增加偏置，降低测量精度。CAT在很大程度上改善了与静态缩尺相关的测量精确度和准确度的下降。该项目将在教育测试和心理测量学领域现有工作的基础上，开发软件和在线网络服务，根据民意研究人员的特定需求采用和调整CAT。本研究项目将CAT算法应用到调查研究领域，并针对民意研究人员的需求提供实践资源和理论指导。CAT算法动态调整以测量潜在结构，同时将每个受访者必须回答的问题数量降至最低。这种方法使用关于每个问题项质量的信息，通过选择后续问题来回答个人先前的答案，这些后续问题将以最高的精度和最少的问题数量将他们置于潜在维度上。也就是说，它根据每个受访者之前的回答为他们选择问题，以便研究人员能够尽可能多地了解受访者的意见。该项目将提供开发CAT电池和对其进行调查管理所需的基本软件基础设施。这包括完成和托管基于云的网络服务，该服务可轻松集成到调查管理软件中。该项目还将为配备自适应电池的调查研究人员可能面临的问题提供解决方案，包括适当处理高水平的测量误差和诊断有缺陷的问题-项目校准。最后，该项目将收集数据，以建立衡量社会科学中重要概念的适应性电池，特别强调个性清单，使用大型便利样本、具有全国代表性的样本和现有数据集的组合。这些校准的分发将允许研究人员使用CAT技术，而不需要增加大量预测试的成本。

This project will advance computerized adaptive testing (CAT) as an alternative approach to static-reduced batteries. A common way that social scientists learn about individuals' traits and beliefs is to have people answer many closely related questions about the same topic. These large multi-item batteries are used to measure constructs like personality traits, psychological illnesses, political ideology, personal values, levels of knowledge, and more. Public opinion researchers, however, often choose not to use these widely respected multi-item batteries because of concerns regarding respondent burden or the costs of long surveys. The standard solution is to select a subset of the available items, which then are administered to all respondents. Such static-reduced batteries are well known to increase bias and lower measurement precision, however. CAT ameliorates much of the decreased measurement precision and accuracy associated with static-reduced scales. The project builds on existing work in the fields of educational testing and psychometrics to develop software and an online webservice that adopts and adjusts CAT for the specific needs of public opinion researchers.This research project will apply CAT algorithms to the domain of survey research and provide practical resources and theoretical guidance specific to the needs of public opinion researchers. CAT algorithms adapt dynamically to measure latent constructs while minimizing the number of questions each respondent must answer. This approach uses information about the qualities of each question item to respond to individuals' prior answers by choosing subsequent questions that will place them on the latent dimension with maximum precision and a minimum number of questions. That is, it chooses questions for each respondent based on their previous responses so that researchers can learn as much as possible about the opinions of a respondent. The project will provide basic software infrastructure necessary for developing CAT batteries and administering them on surveys. This includes the completion and hosting of a cloud-based webservice easily integrated into survey-administration software. The project also will provide solutions to problems likely to confront survey researchers fielding adaptive batteries including appropriately handling high levels of measurement error and diagnosing flawed question-item calibrations. Finally, the project will collect data to build adaptive batteries measuring important concepts in the social sciences, with a special emphasis on personality inventories, using a combination of large convenience samples, a nationally representative sample, and existing datasets. The distribution of these calibrations will allow researchers to use CAT techniques without the added cost of extensive pre-testing.