Evaluation-Based Effectiveness Testing

描述(由申请人提供)：这项SBIR的目标是开发一个在线工具，合成控制，以帮助当地评估者和其他负责证明毒品预防计划有效性的人获得证据，以评估其干预措施的有效性。综合控制将提供有用的分析，同时减轻当地评估者必须确定、召集、维持和管理对对照小组的调查的负担。该系统将元分析地结合以前管理的调查的存档数据--包括来自相关项目的干预小组和可用的控制组的评估数据--为当地评估者建立反事实。因此，这种方法的目的不是依靠黄金标准方法(例如随机对照试验)来评估影响，而是提供一种统计上可靠的方法来确定地方方案的影响，方法是将当前的绩效与先前评估的累积数据库设定的基准进行比较。在第一阶段，我们能够证明使用现有数据库完成拟议方法的试点测试的可行性。我们改进了使用参考数据完成分析的统计方法。我们完成了广泛的分析，以证明所提出的方法的可行性和实用性。在第二阶段，我们建议完成以下任务：1.我们将设计一个数据库，有效地重组各种数据配置，使用户能够选择适当的比较组。2.我们将开发数据分析算法，为行为变量和中介变量分析构建适当的比较。3.我们将制定协议，自动将应用程序与使用评估蜥蜴进行的当地评估收集的数据联系起来。4.我们将制定协议，提供一种方法，以导入使用其他(而不是评价蜥蜴)服务进行的地方评价收集的数据。5.我们将设计一个合成控制用户界面，允许客户注册、管理项目、完成分析和下载报告。6.我们将进行阿尔法测试，以调试系统。7.我们将进行测试版测试，以证明该系统可以与各种潜在客户一起工作。 公共卫生相关性：目前，地方对毒品预防方案的评估缺乏评估有效性的能力，因为很少有(如果有的话)包括对照或对照小组，以确定他们降低药物使用始发率的程度。该项目将提供一种创新战略，使地方毒品预防评价能够将其评价结果与以前的地方评价结果进行比较。这将使项目能够评估它们的相对有效性。

DESCRIPTION (provided by applicant): The goal of this SBIR is to develop an online tool, Synthetic Controls, to help local evaluators and others charged with demonstrating evidence of effectiveness of drug prevention programs obtain evidence for assessing the effectiveness of their interventions. Synthetic Controls will provide useful analyses and simultaneously relieve local evaluators of the burden of having to identify, convene, maintain, and administer surveys to a comparison group. The system will meta-analytically combine archived data from previously administered surveys - including evaluation data from intervention groups of relevant programs and from control groups where available - to establish counterfactuals for local evaluators. Thus, rather than relying on gold-standard methods (e.g. randomized control trials) for assessing impact, the goal of this approach will be to provide a statistically reliable method for determining the impact of local programs by comparing current performance against benchmarks set by an accumulated database of previous evaluations. During Phase I, we were able to demonstrate the feasibility of using an extant database for completing a pilot test of proposed methods. We refined statistical methods for completing analyses using referent data. We completed extensive analyses to demonstrate the feasibility and utility of the proposed methods. During Phase II, we propose to complete the following tasks: 1. We will design a database that will efficiently reconstitute various configurations of data to allow users to select appropriate comparison groups. 2. We will develop data analysis algorithms that will construct appropriate comparisons for behavioral and mediating variable analyses. 3. We will develop protocols to automatically link the application to data collected by local evaluations conducted using Evaluation Lizard. 4. We will develop protocols to provide a method to import data collected by local evaluations conducted using other (not Evaluation Lizard) services. 5. We will design a Synthetic Controls user interface that will allow clients to register, manage projects, complete analyses, and download reports. 6. We will conduct an alpha test to debug the system. 7. We will conduct a beta test to demonstrate that the system will work with a variety of potential clients. PUBLIC HEALTH RELEVANCE: Currently, local evaluations of drug prevention programs lack the ability to assess effectiveness because few, if any, include control or comparison groups against which to establish the degree to which they have reduced the rate of drug use onset. This project will provide an innovative strategy whereby local drug prevention evaluations can compare their findings with the findings of previous local evaluations. This will allow programs to assess their relative effectiveness.