New CAT Research Tool:IRT/CAT Software for Secure Web & IVR Survey Administration

描述(由申请人提供)：计算机自适应测试(CAT)为评估患者报告的结果提供了一种新的机制。使用CAT可以：(A)提高测量精度，从而减少实现统计显著结果所需的参与者数量；(B)与实施固定长度调查相比，减轻受访者的测试负担，从而增加愿意参与研究的潜在参与者数量；以及(C)为评估各种治疗方式的有效性提供一个共同的衡量标准。所有这些能力将加快临床干预有效性研究向临床护理的转移。非常成功的NIH患者报告结果测量信息系统(PROIS)路线图计划开发了一套经过充分验证的基于IRT的临床症状和功能题库，以及一个用于管理固定长度和CAT调查项目的网站。然而，PROMIS评估中心不符合21-CFR-PART-11，因此不能用于FDA的临床试验。此外，它目前不允许创建交互式语音应答(IVR)调查，也不允许客户私人存储数据或在公司网站上运行调查。事实上，目前还没有可以部署在公司服务器上的软件包，可以使用CAT算法管理Web或IVR调查。除非开发出具有21-CFR-Part-11兼容CAT功能的商业调查软件，否则PROMIS倡议的好处可能无法实现。该提案的总体目标是创建一个CAT软件模块(“SmartCATTM”)，使用FDA标准的21-CFR-Part-11最佳实践，可以集成到现有的调查软件产品中。我们将在两位杰出的IRT-CAT研究人员Seung Choi博士和Steve Reise博士的帮助下开发SmartCATTM。SmartCATTM将通过正式设计审查、手动和自动操作测试、根据现有标准验证实际算法操作以及与潜在客户进行两轮完整的现场测试来进行测试。SmartCATTM还将集成到现有的两个TeleSage软件包中：(1)用于网络调查管理的SmartQ-WebTM和(2)用于自动电话调查管理的SmartQ-IVRTM，以及由美国访谈服务公司和SurveyGizmo创建的软件包，这两家大型调查研究公司是我们在本项目中的合作伙伴。一旦成功集成到所有这四个程序中，我们将开发和集成高级CAT功能，例如内容平衡和敌方项目控制。由此产生的TeleSage SmartCATTM和SmartQ调查套件将为CAT调查管理提供极大的灵活性，从而在降低受访者负担的同时提高测量精度。基于这些原因，以及由于拟议的一揽子软件将使私营部门能够广泛使用现有的PROIS题库，我们相信这可能是一个影响很大的项目。 与公共健康相关：建议在此应用程序中开发的CAT调查管理软件商业套件，即SmartCATTM、SmartQ-Web-CATTM和SmartQ-IVR-CATTM，将显著减少患者完成症状和功能评估所需的时间，从而在不显著损失测量精度的情况下减轻患者受访者的测试负担。将CAT调查软件产品提供给包括商业实体在内的更广泛的受众，将加速将研究转化为临床护理，并促进实现NIH PROMIS路线图倡议的目标。

DESCRIPTION (provided by applicant): Computer Adaptive Testing (CAT) provides a new mechanism for assessing patient-reported outcomes. Using CAT can: (a) increase measurement precision, leading to a decrease in the number of participants necessary to achieve statistically significant results, (b) decrease test burden on respondents, compared to administration of fixed length surveys, thereby increasing the potential number of participants willing to participate in research, and (c) provide a common metric for assessing the effectiveness of varying treatment modalities. All of these capabilities will speed the transfer of clinical intervention effectiveness research into clinical care. The highly successful NIH Patient Reported Outcomes Measurement Information System (PROMIS) roadmap initiative has developed a diverse set of well-validated IRT-based clinical symptoms and functioning item banks as well as a web site for administering the items as fixed-length and CAT surveys. However, the PROMIS Assessment Center is not "21-CFR-Part-11" compliant, and therefore cannot be used in FDA clinical trials. In addition, it does not currently permit the creation of Interactive Voice Response (IVR) surveys, nor does it allow clients to store data privately or run surveys on a corporate website. In fact, at present, no software package is available that can be deployed on a corporate server to administer web or IVR surveys using CAT algorithms. Unless commercial survey software with 21-CFR-Part-11 compliant CAT functionality is developed, the benefits of the PROMIS initiative may not be realized. The overarching goal of this proposal is to create a CAT software module ("SmartCATTM"), using FDA- standard 21-CFR-Part-11 Best Practices, that can be integrated into existing survey software products. We will develop SmartCATTM with assistance from Seung Choi, PhD, and Steve Reise, PhD, two outstanding IRT-CAT researchers. SmartCATTM will be tested via formal design review, manual and automated operational testing, validation of actual algorithmic operations against existing standards, and two full rounds of field testing with potential customers. SmartCATTM will also be integrated into two existing TeleSage software packages: (1) SmartQ-WebTM for web survey administration and (2) SmartQ-IVRTM for automated telephone survey administration, as well as software packages created by Interviewing Services of America and SurveyGizmo, two large survey research companies with whom we have partnered for this project. Once successful integration into all four of these programs has been demonstrated, we will develop and integrate advanced CAT features, e.g., content balancing and enemy item control. The resulting TeleSage SmartCATTM and SmartQ survey suite will provide tremendous flexibility for CAT survey administration, thereby promoting greater measurement precision while decreasing respondent burden. For these reasons and because the proposed software package will make widespread private sector use of the existing PROMIS item banks possible, we believe that this is likely to be a high impact project. PUBLIC HEALTH RELEVANCE: The commercial suite of CAT survey administration software products proposed for development in this application, namely SmartCATTM, SmartQ-Web-CATTM, and SmartQ-IVR-CATTM, will afford significant reductions in the time required for patients to complete symptom and functioning assessments, thereby reducing the test burden on patient respondents, without significant loss of measurement precision. Making CAT survey software products available to a wider audience, including commercial entities, will speed the translation of research into clinical care and facilitate the realization of the goals of the NIH PROMIS roadmap initiative.