A flexible, web-based system for design, support, and delivery of adaptive tests

描述（由申请人提供）：在行为研究界，对项目反应理论框架（IRT）中计算机化自适应测试（CAT）的巨大优势的认识正在迅速增长。CAT有能力在不损失测量可靠性的情况下显著减少应答者负担。虽然存在少量用于行为测量的计算机化自适应测试（CATs），但它们被设计为“一次性”cat，用于评估一组固定的领域。除了最初的开发人员之外，目前可用的cat不能被个人轻易地改变以合并新域，这一事实严重限制了cat的可用性，超出了它们最初的目的。因此，尽管人们越来越意识到这种最先进的方法，但对于大多数行为研究人员来说，目前在他们的研究设计中实施CAT是不可行的。CATs的构建和维护需要具有广泛心理测量学和计算机经验的个人持续关注。为数不多的用于创建CAT的可用软件产品也极为有限：它们是基于pc的，因此必须在用于CAT管理的每台计算机上部署必要的软件；一旦在个人电脑上安装了软件，就必须把这台电脑带到参与者面前，或者把参与者带到个人电脑面前；后续的软件更新需要互联网连接或物理介质；这些项目有广泛的系统要求。拟议的项目旨在克服CAT的这些障碍，使CAT成为任何可以访问互联网的行为研究人员可用的工具。[在第二阶段完成时]拟议项目将提供一个灵活的基于网络的CAT系统，以便任何研究人员可以登录到一个网站并创建一个适应性测试。作为最终目标，该系统将足够灵活，如果需要，用户可以在没有VPG帮助的情况下远程创建和管理CATs。然而，希望使用该系统的研究人员将有VPG的工作人员可以帮助他们创建和管理CATs，并将结果测量纳入他们的研究设计。第一阶段的完成将产生一个基于web的CAT系统原型。这个[基本]原型将被设计为通过上传文件或直接用户输入来接收用户输入，将用户输入合并到使用IRT模型实现的CAT算法中，管理自适应测试，并为用户直接打印或下载生成输出。[第一阶段的重点是实现核心CAT算法和基本支持输入/输出系统作为动态web应用程序。拟议项目的第二阶段将征求广泛潜在用户的反馈，以扩展原型，重点关注可用性和持续的灵活性，并将包括试点测试和改进，以确保基于网络的CAT系统和支持服务的设计完全满足行为研究和评估社区的需求。公共卫生相关性：

DESCRIPTION (provided by applicant): Awareness of the enormous advantages of computerized adaptive testing (CAT) in an item response theory framework (IRT) is growing rapidly in the behavioral research community. CAT has the capacity to significantly reduce respondent burden without losing measurement reliability. Although a small number of computerized adaptive tests (CATs) exist for behavioral measurement, they have been designed as "single-use" CATs which were developed to assess a fixed set of domains. The fact that currently available CATs cannot be easily altered by individuals other than the original developers to incorporate new domains severely limits the usability of the CATs beyond their initial purpose. Thus, despite the growing awareness of this state-of-the-art methodology, it is not currently feasible for the majority of behavioral researchers to implement CAT in their research designs. The construction and maintenance of CATs requires ongoing attention by individuals with extensive psychometric and computing experience. The few available software products for creating CATs are also extremely limited: they are PC-based so necessary software must be deployed on each computer that is going to be used for CAT administration; once software is installed on a PC that PC must be taken to the participants or the participants must be brought to the PC; subsequent software updates require internet connectivity or physical media; and these programs have extensive system requirements. The proposed project aims to overcome these barriers to CAT, making CAT an available tool for any behavioral researcher with access to the internet. [At the completion of Phase II] of the proposed project, a flexible web-based CAT system will be available so that any researcher can log-on to a web-site and create an adaptive test. [As a final goal,] the system will be flexible enough that users can, if desired, create and administer CATs remotely and without VPG assistance. However, researchers wishing to use this system will have the VPG staff available to them to help create and administer CATs, and to incorporate the resulting measures into their research design. The completion of Phase I will result in a prototype web-based CAT system. This [basic] prototype will be designed to receive input from a user through upload of files or direct user entry, incorporate the user input into a CAT algorithm implemented with IRT models, administer an adaptive test, and generate output for direct printing or download by the user. [The focus of Phase I is on the implementation of the core CAT algorithms and basic support input/output systems as dynamic web applications.]Phase II of the proposed project will solicit feedback from a wide array of potential users to expand the prototype, focusing on usability and continued flexibility, and will include pilot testing and refinement to ensure the web-based CAT system and support services are designed to fully meet the needs of behavioral research and assessment communities. PUBLIC HEALTH RELEVANCE: Computerized adaptive testing (CAT) has the capacity to significantly reduce respondent burden without any loss of reliability. However, CAT technology is currently accessible only to those with extensive psychometric knowledge thereby limiting the majority of behavioral researchers from implementing CAT in their own research. The proposed project aims to overcome these barriers to access by creating a flexible web-based CAT system that will allow behavioral researchers to incorporate this state-of-the-art methodology into their study designs.