Computerized Adaptive Testing for Survey Research

用于调查研究的计算机化自适应测试

• 批准号: 1558907
• 负责人: Jacob Montgomery
• 金额: $ 29.16万
• 依托单位: Washington University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-04-15 至 2021-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1558907&HistoricalAwards=false
• 关键词: Computerized; Adaptive; Testing; Survey; Research

This project will advance computerized adaptive testing (CAT) as an alternative approach to static-reduced batteries. A common way that social scientists learn about individuals' traits and beliefs is to have people answer many closely related questions about the same topic. These large multi-item batteries are used to measure constructs like personality traits, psychological illnesses, political ideology, personal values, levels of knowledge, and more. Public opinion researchers, however, often choose not to use these widely respected multi-item batteries because of concerns regarding respondent burden or the costs of long surveys. The standard solution is to select a subset of the available items, which then are administered to all respondents. Such static-reduced batteries are well known to increase bias and lower measurement precision, however. CAT ameliorates much of the decreased measurement precision and accuracy associated with static-reduced scales. The project builds on existing work in the fields of educational testing and psychometrics to develop software and an online webservice that adopts and adjusts CAT for the specific needs of public opinion researchers.This research project will apply CAT algorithms to the domain of survey research and provide practical resources and theoretical guidance specific to the needs of public opinion researchers. CAT algorithms adapt dynamically to measure latent constructs while minimizing the number of questions each respondent must answer. This approach uses information about the qualities of each question item to respond to individuals' prior answers by choosing subsequent questions that will place them on the latent dimension with maximum precision and a minimum number of questions. That is, it chooses questions for each respondent based on their previous responses so that researchers can learn as much as possible about the opinions of a respondent. The project will provide basic software infrastructure necessary for developing CAT batteries and administering them on surveys. This includes the completion and hosting of a cloud-based webservice easily integrated into survey-administration software. The project also will provide solutions to problems likely to confront survey researchers fielding adaptive batteries including appropriately handling high levels of measurement error and diagnosing flawed question-item calibrations. Finally, the project will collect data to build adaptive batteries measuring important concepts in the social sciences, with a special emphasis on personality inventories, using a combination of large convenience samples, a nationally representative sample, and existing datasets. The distribution of these calibrations will allow researchers to use CAT techniques without the added cost of extensive pre-testing.

该项目将推进计算机化自适应测试（CAT），作为减少静电电池的替代方法。 社会科学家了解个人特征和信仰的一种常见方法是让人们回答许多关于同一主题的密切相关的问题。这些大型的多项目电池被用来衡量人格特质，心理疾病，政治意识形态，个人价值观，知识水平等结构。 然而，民意研究人员往往选择不使用这些广受推崇的多项目电池，因为担心受访者的负担或长期调查的成本。 标准的解决方案是选择可用项目的一个子集，然后将其管理给所有受访者。 然而，众所周知，这种静电减少的电池会增加偏置并降低测量精度。 CAT改善了与静态缩减尺度相关的测量精度和准确度下降。 该项目以教育测试和心理测量领域的现有工作为基础，开发软件和在线网络服务，根据民意研究人员的特定需求采用和调整CAT。该研究项目将CAT算法应用于调查研究领域，并提供针对民意研究人员需求的实践资源和理论指导。 CAT算法动态地适应测量潜在的结构，同时最大限度地减少每个受访者必须回答的问题的数量。 这种方法使用关于每个问题项的质量的信息，通过选择后续问题来响应个体的先前答案，这些后续问题将以最大的精度和最少的问题数量将它们置于潜在维度上。也就是说，它根据每个受访者之前的回答为他们选择问题，以便研究人员可以尽可能多地了解受访者的意见。 该项目将提供开发计算机辅助测试成套设备和在调查中管理这些设备所需的基本软件基础设施。这包括完成和托管一个基于云的网络服务，可以轻松集成到调查管理软件中。 该项目还将为调查研究人员可能面临的问题提供解决方案，包括适当处理高水平的测量误差和诊断有缺陷的问题项目校准。最后，该项目将收集数据，以建立适应性电池，测量社会科学中的重要概念，特别强调个性清单，使用大型便利样本，全国代表性样本和现有数据集的组合。 这些校准的分布将使研究人员能够使用CAT技术，而无需增加大量预测试的成本。