New CAT Research Tool:IRT/CAT Software for Secure Web & IVR Survey Administration

描述（由申请人提供）：计算机自适应测试（CAT）提供了一种评估患者报告结果的新机制。使用CAT可以：(a)提高测量精度，导致获得统计显著结果所需的参与者人数减少；(b)与固定长度调查的管理相比，减少受访者的测试负担，从而增加愿意参与研究的潜在参与者人数；(c)为评估不同治疗方式的有效性提供一个共同的指标。所有这些能力将加速临床干预有效性研究向临床护理的转移。非常成功的美国国立卫生研究院患者报告结果测量信息系统（PROMIS）路线图倡议已经开发了一套多样化的、经过良好验证的基于irt的临床症状和功能物题库，以及一个管理固定长度和CAT调查项目的网站。然而，PROMIS评估中心不符合“21-CFR-Part-11”标准，因此不能用于FDA临床试验。此外，它目前不允许创建交互式语音应答（IVR）调查，也不允许客户私下存储数据或在公司网站上进行调查。事实上，目前还没有可以部署在企业服务器上的软件包，可以使用CAT算法来管理web或IVR调查。除非开发出符合21-CFR-Part-11 CAT功能的商业调查软件，否则PROMIS计划的好处可能无法实现。本提案的总体目标是创建一个CAT软件模块（“SmartCATTM”），使用FDA标准21-CFR-Part-11最佳实践，可以集成到现有的调查软件产品中。我们将在Seung Choi博士和Steve Reise博士两位杰出的IRT-CAT研究人员的协助下开发SmartCATTM。SmartCATTM将通过正式的设计审查、手动和自动操作测试、根据现有标准验证实际算法操作，以及与潜在客户进行两轮完整的现场测试来进行测试。SmartCATTM还将集成到现有的两个TeleSage软件包中：(1)用于网络调查管理的SmartQ-WebTM和(2)用于自动电话调查管理的SmartQ-IVRTM，以及由美国访谈服务公司和SurveyGizmo开发的软件包，这两家大型调查研究公司是我们在这个项目上的合作伙伴。一旦成功集成到所有这四个项目中，我们将开发和集成先进的CAT功能，例如内容平衡和敌人物品控制。由此产生的TeleSage SmartCATTM和SmartQ调查套件将为CAT调查管理提供巨大的灵活性，从而提高测量精度，同时减少被调查者的负担。由于这些原因，并且由于拟议的软件包将使私营部门能够广泛使用现有的PROMIS物项库，我们认为这可能是一个影响很大的项目。

DESCRIPTION (provided by applicant): Computer Adaptive Testing (CAT) provides a new mechanism for assessing patient-reported outcomes. Using CAT can: (a) increase measurement precision, leading to a decrease in the number of participants necessary to achieve statistically significant results, (b) decrease test burden on respondents, compared to administration of fixed length surveys, thereby increasing the potential number of participants willing to participate in research, and (c) provide a common metric for assessing the effectiveness of varying treatment modalities. All of these capabilities will speed the transfer of clinical intervention effectiveness research into clinical care. The highly successful NIH Patient Reported Outcomes Measurement Information System (PROMIS) roadmap initiative has developed a diverse set of well-validated IRT-based clinical symptoms and functioning item banks as well as a web site for administering the items as fixed-length and CAT surveys. However, the PROMIS Assessment Center is not "21-CFR-Part-11" compliant, and therefore cannot be used in FDA clinical trials. In addition, it does not currently permit the creation of Interactive Voice Response (IVR) surveys, nor does it allow clients to store data privately or run surveys on a corporate website. In fact, at present, no software package is available that can be deployed on a corporate server to administer web or IVR surveys using CAT algorithms. Unless commercial survey software with 21-CFR-Part-11 compliant CAT functionality is developed, the benefits of the PROMIS initiative may not be realized. The overarching goal of this proposal is to create a CAT software module ("SmartCATTM"), using FDA- standard 21-CFR-Part-11 Best Practices, that can be integrated into existing survey software products. We will develop SmartCATTM with assistance from Seung Choi, PhD, and Steve Reise, PhD, two outstanding IRT-CAT researchers. SmartCATTM will be tested via formal design review, manual and automated operational testing, validation of actual algorithmic operations against existing standards, and two full rounds of field testing with potential customers. SmartCATTM will also be integrated into two existing TeleSage software packages: (1) SmartQ-WebTM for web survey administration and (2) SmartQ-IVRTM for automated telephone survey administration, as well as software packages created by Interviewing Services of America and SurveyGizmo, two large survey research companies with whom we have partnered for this project. Once successful integration into all four of these programs has been demonstrated, we will develop and integrate advanced CAT features, e.g., content balancing and enemy item control. The resulting TeleSage SmartCATTM and SmartQ survey suite will provide tremendous flexibility for CAT survey administration, thereby promoting greater measurement precision while decreasing respondent burden. For these reasons and because the proposed software package will make widespread private sector use of the existing PROMIS item banks possible, we believe that this is likely to be a high impact project. PUBLIC HEALTH RELEVANCE: The commercial suite of CAT survey administration software products proposed for development in this application, namely SmartCATTM, SmartQ-Web-CATTM, and SmartQ-IVR-CATTM, will afford significant reductions in the time required for patients to complete symptom and functioning assessments, thereby reducing the test burden on patient respondents, without significant loss of measurement precision. Making CAT survey software products available to a wider audience, including commercial entities, will speed the translation of research into clinical care and facilitate the realization of the goals of the NIH PROMIS roadmap initiative.