Evaluation-Based Effectiveness Testing

基于评估的有效性测试

• 批准号: 8123884
• 负责人: William B. Hansen
• 金额: $ 49.89万
• 依托单位: TANGLEWOOD RESEARCH, INC.
• 依托单位国家: 美国
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-06-01 至 2013-05-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/8123884
• 关键词: Algorithms; Archives; Behavioral; Benchmarking; Charge; Client; Communities; Community Services; Control Groups; Data; Data Analyses; Data Reporting; Databases; Development; Drug Prevention Program; Drug usage; Education; Effectiveness; Evaluation; Funding; Goals; Gold; Intervention; Link; Lizards; Mediating; Meta-Analysis; Methods; Online Systems; Outcome; Participant; Performance; Pharmaceutical Preparations; Phase; Policy Maker; Prevention; Prevention program; Program Evaluation; Protocols documentation; Provider; Randomized Controlled Trials; Registries; Relative (related person); Reporting; Research; Research Personnel; Schools; Services; Small Business Innovation Research Grant; Statistical Methods; Sum; Surveys; System; Testing; United States Substance Abuse and Mental Health Services Administration; Work; base; comparison group; design; drug abuse prevention; evidence base; group intervention; innovation; prevention evaluation; programs; reconstitution; tool

DESCRIPTION (provided by applicant): The goal of this SBIR is to develop an online tool, Synthetic Controls, to help local evaluators and others charged with demonstrating evidence of effectiveness of drug prevention programs obtain evidence for assessing the effectiveness of their interventions. Synthetic Controls will provide useful analyses and simultaneously relieve local evaluators of the burden of having to identify, convene, maintain, and administer surveys to a comparison group. The system will meta-analytically combine archived data from previously administered surveys - including evaluation data from intervention groups of relevant programs and from control groups where available - to establish counterfactuals for local evaluators. Thus, rather than relying on gold-standard methods (e.g. randomized control trials) for assessing impact, the goal of this approach will be to provide a statistically reliable method for determining the impact of local programs by comparing current performance against benchmarks set by an accumulated database of previous evaluations. During Phase I, we were able to demonstrate the feasibility of using an extant database for completing a pilot test of proposed methods. We refined statistical methods for completing analyses using referent data. We completed extensive analyses to demonstrate the feasibility and utility of the proposed methods. During Phase II, we propose to complete the following tasks: 1. We will design a database that will efficiently reconstitute various configurations of data to allow users to select appropriate comparison groups. 2. We will develop data analysis algorithms that will construct appropriate comparisons for behavioral and mediating variable analyses. 3. We will develop protocols to automatically link the application to data collected by local evaluations conducted using Evaluation Lizard. 4. We will develop protocols to provide a method to import data collected by local evaluations conducted using other (not Evaluation Lizard) services. 5. We will design a Synthetic Controls user interface that will allow clients to register, manage projects, complete analyses, and download reports. 6. We will conduct an alpha test to debug the system. 7. We will conduct a beta test to demonstrate that the system will work with a variety of potential clients. PUBLIC HEALTH RELEVANCE: Currently, local evaluations of drug prevention programs lack the ability to assess effectiveness because few, if any, include control or comparison groups against which to establish the degree to which they have reduced the rate of drug use onset. This project will provide an innovative strategy whereby local drug prevention evaluations can compare their findings with the findings of previous local evaluations. This will allow programs to assess their relative effectiveness.

描述（由申请人提供）：本SBIR的目标是开发一个在线工具，合成控制，以帮助当地评估人员和其他负责证明药物预防计划有效性的人获得评估其干预措施有效性的证据。综合控制将提供有用的分析，同时减轻地方评价人员必须确定、召集、维持和管理对比较组的调查的负担。该系统将对以前管理的调查的存档数据进行荟萃分析，包括来自相关项目干预组和控制组的评估数据，以便为当地评估人员建立反事实。因此，与其依赖黄金标准方法（例如，随机对照试验）来评估影响，这种方法的目标将是提供一种统计上可靠的方法，通过将当前绩效与以往评估累积数据库设定的基准进行比较，来确定地方项目的影响。在第一阶段，我们能够证明使用现有数据库完成所提议方法的试点测试的可行性。我们改进了统计方法，以便利用参考数据完成分析。我们完成了广泛的分析，以证明所提出方法的可行性和实用性。在第二阶段，我们建议完成以下任务：1。我们将设计一个数据库，将有效地重建各种配置的数据，让用户选择适当的比较组。2. 我们将开发数据分析算法，为行为和中介变量分析构建适当的比较。3. 我们将开发协议，将应用程序自动链接到使用Evaluation Lizard进行的本地评估收集的数据。我们将开发协议，以提供一种方法来导入使用其他（不是Evaluation Lizard）服务进行的本地评估收集的数据。5. 我们将设计一个Synthetic Controls用户界面，允许客户注册、管理项目、完成分析和下载报告。6. 我们将进行一次alpha测试来调试系统。7. 我们将进行beta测试，以证明该系统将与各种潜在客户一起工作。