Evaluation-Based Effectiveness Testing

基于评估的有效性测试

基本信息

批准号：
8248162
负责人：
William B. Hansen
金额：
$ 48.42万
依托单位：
TANGLEWOOD RESEARCH, INC.
依托单位国家：
美国
项目类别：
财政年份：
2011
资助国家：
美国
起止时间：
2011-06-01 至 2014-05-31
项目状态：
已结题

来源：
https://reporter.nih.gov/project-details/8248162
关键词：
Algorithms Archives Behavioral Benchmarking Charge Client Communities Community Services Control Groups Data Data Analyses Data Reporting Databases Development Drug Prevention Program Drug usage Education Effectiveness Evaluation Funding Goals Gold Intervention Link Lizards Mediating Meta-Analysis Methods Online Systems Outcome Participant Performance Pharmaceutical Preparations Phase Policy Maker Prevention Prevention program Program Evaluation Protocols documentation Provider Randomized Controlled Trials Registries Relative (related person)Reporting Research Research Personnel Schools Services Small Business Innovation Research Grant Statistical Methods Sum Surveys System Testing United States Substance Abuse and Mental Health Services Administration Work base comparison group design drug abuse prevention evidence base group intervention innovation prevention evaluation programs public health relevance reconstitution tool

项目摘要

DESCRIPTION (provided by applicant): The goal of this SBIR is to develop an online tool, Synthetic Controls, to help local evaluators and others charged with demonstrating evidence of effectiveness of drug prevention programs obtain evidence for assessing the effectiveness of their interventions. Synthetic Controls will provide useful analyses and simultaneously relieve local evaluators of the burden of having to identify, convene, maintain, and administer surveys to a comparison group. The system will meta-analytically combine archived data from previously administered surveys - including evaluation data from intervention groups of relevant programs and from control groups where available - to establish counterfactuals for local evaluators. Thus, rather than relying on gold-standard methods (e.g. randomized control trials) for assessing impact, the goal of this approach will be to provide a statistically reliable method for determining the impact of local programs by comparing current performance against benchmarks set by an accumulated database of previous evaluations. During Phase I, we were able to demonstrate the feasibility of using an extant database for completing a pilot test of proposed methods. We refined statistical methods for completing analyses using referent data. We completed extensive analyses to demonstrate the feasibility and utility of the proposed methods. During Phase II, we propose to complete the following tasks: 1. We will design a database that will efficiently reconstitute various configurations of data to allow users to select appropriate comparison groups. 2. We will develop data analysis algorithms that will construct appropriate comparisons for behavioral and mediating variable analyses. 3. We will develop protocols to automatically link the application to data collected by local evaluations conducted using Evaluation Lizard. 4. We will develop protocols to provide a method to import data collected by local evaluations conducted using other (not Evaluation Lizard) services. 5. We will design a Synthetic Controls user interface that will allow clients to register, manage projects, complete analyses, and download reports. 6. We will conduct an alpha test to debug the system. 7. We will conduct a beta test to demonstrate that the system will work with a variety of potential clients. PUBLIC HEALTH RELEVANCE: Currently, local evaluations of drug prevention programs lack the ability to assess effectiveness because few, if any, include control or comparison groups against which to establish the degree to which they have reduced the rate of drug use onset. This project will provide an innovative strategy whereby local drug prevention evaluations can compare their findings with the findings of previous local evaluations. This will allow programs to assess their relative effectiveness.

描述（由申请人提供）：该SBIR的目的是开发一种在线工具，合成控制，以帮助当地评估者和其他负责证明药物预防计划的证据的人获得评估其干预效果的证据。合成控制将提供有用的分析，并同时减轻本地评估者必须识别，召集，维护和管理对比较组的调查的负担。该系统将荟萃分析结合先前管理的调查的存档数据 - 包括来自相关程序的干预组和可用对照组的评估数据，以建立本地评估者的反事实。因此，该方法的目的不是依靠金标准方法（例如随机对照试验）来评估影响，而是提供一种统计上可靠的方法来确定局部程序的影响，通过将当前程序与先前评估的累积数据库设置的基准进行比较。在第一阶段，我们能够证明使用现存数据库完成提议方法的试验测试的可行性。我们完善了使用参考数据来完成分析的统计方法。我们完成了广泛的分析，以证明所提出方法的可行性和实用性。在第二阶段，我们建议完成以下任务：1。我们将设计一个数据库，该数据库将有效地重建数据的各种配置，以允许用户选择适当的比较组。 2。我们将开发数据分析算法，以构建适当的比较行为和中介变量分析。 3。我们将开发协议，以自动将应用程序与使用评估蜥蜴进行的本地评估收集的数据联系起来。 4。我们将开发协议，以提供一种通过使用其他（而非评估蜥蜴）服务进行的本地评估收集的数据的方法。 5。我们将设计一个综合控件用户界面，该界面将允许客户注册，管理项目，完成分析和下载报告。 6。我们将进行Alpha测试以调试系统。 7。我们将进行Beta测试，以证明该系统将与各种潜在客户合作。公共卫生相关性：目前，对药物预防计划的本地评估缺乏评估有效性的能力，因为很少（如果有的话）包括控制或比较组，以确定他们降低药物使用率的程度。该项目将提供一种创新的策略，在该策略中，本地药物预防评估可以将其发现与以前的本地评估的发现进行比较。这将允许程序评估其相对有效性。