Evaluation-Based Effectiveness Testing

描述（由申请人提供）：该SBIR的目标是开发一种在线工具，合成控制，以帮助当地评估人员和其他负责证明药物预防计划有效性的人员获得评估其干预措施有效性的证据。综合控制将提供有用的分析，同时减轻当地评估人员必须识别、召集、维护和管理对照组调查的负担。该系统将以元分析的方式将以前管理的调查中的存档数据（包括相关方案的干预组和对照组（如有）的评估数据）联合收割机结合起来，为当地评估人员建立反事实。因此，这种方法的目标不是依靠金标准方法（例如随机对照试验）来评估影响，而是提供一种统计上可靠的方法，通过将当前的绩效与以往评估的累积数据库所设定的基准进行比较，来确定地方方案的影响。 在第一阶段，我们能够证明使用现有的数据库来完成所提出的方法的试点测试的可行性。我们改进了统计方法，使用参考数据完成分析。我们完成了广泛的分析，以证明所提出的方法的可行性和实用性。 在第二阶段，我们建议完成以下工作：我们将设计一个数据库，它将有效地重建各种数据配置，以允许用户选择适当的比较组。 2.我们将开发数据分析算法，为行为和中介变量分析构建适当的比较。 3.我们将开发协议，以自动将应用程序与使用Evaluation Lounge进行的本地评估收集的数据联系起来。 4.我们将制定协议，以提供一种方法来导入使用其他（非评估许可证）服务进行的本地评估所收集的数据。 5.我们将设计一个合成控件用户界面，允许客户注册，管理项目，完成分析和下载报告。 6.我们将进行Alpha测试来调试系统。 7.我们将进行beta测试，以证明该系统将与各种潜在的客户。 公共卫生相关性：目前，对毒品预防方案的地方评估缺乏评估有效性的能力，因为很少（如果有的话）包括对照组或比较组，以确定它们在多大程度上降低了毒品使用率。该项目将提供一种创新战略，使地方预防吸毒评价能够将其评价结果与以前地方评价的结果进行比较。这将允许程序评估其相对有效性。

DESCRIPTION (provided by applicant): The goal of this SBIR is to develop an online tool, Synthetic Controls, to help local evaluators and others charged with demonstrating evidence of effectiveness of drug prevention programs obtain evidence for assessing the effectiveness of their interventions. Synthetic Controls will provide useful analyses and simultaneously relieve local evaluators of the burden of having to identify, convene, maintain, and administer surveys to a comparison group. The system will meta-analytically combine archived data from previously administered surveys - including evaluation data from intervention groups of relevant programs and from control groups where available - to establish counterfactuals for local evaluators. Thus, rather than relying on gold-standard methods (e.g. randomized control trials) for assessing impact, the goal of this approach will be to provide a statistically reliable method for determining the impact of local programs by comparing current performance against benchmarks set by an accumulated database of previous evaluations. During Phase I, we were able to demonstrate the feasibility of using an extant database for completing a pilot test of proposed methods. We refined statistical methods for completing analyses using referent data. We completed extensive analyses to demonstrate the feasibility and utility of the proposed methods. During Phase II, we propose to complete the following tasks: 1. We will design a database that will efficiently reconstitute various configurations of data to allow users to select appropriate comparison groups. 2. We will develop data analysis algorithms that will construct appropriate comparisons for behavioral and mediating variable analyses. 3. We will develop protocols to automatically link the application to data collected by local evaluations conducted using Evaluation Lizard. 4. We will develop protocols to provide a method to import data collected by local evaluations conducted using other (not Evaluation Lizard) services. 5. We will design a Synthetic Controls user interface that will allow clients to register, manage projects, complete analyses, and download reports. 6. We will conduct an alpha test to debug the system. 7. We will conduct a beta test to demonstrate that the system will work with a variety of potential clients. PUBLIC HEALTH RELEVANCE: Currently, local evaluations of drug prevention programs lack the ability to assess effectiveness because few, if any, include control or comparison groups against which to establish the degree to which they have reduced the rate of drug use onset. This project will provide an innovative strategy whereby local drug prevention evaluations can compare their findings with the findings of previous local evaluations. This will allow programs to assess their relative effectiveness.