Measuring the Quality of Delivery of Prevention

描述(由申请人提供)：众所周知，提供的质量会降低药物滥用预防计划的有效性。随着基于研究的项目得到广泛传播，拥有用于评估交付质量的工具对于记录和了解项目成功和失败的条件变得重要。目前还没有标准方法来衡量提供预防方案的质量。该项目的目标是创建一种标准化工具，用于评估基于研究的药物滥用预防计划的交付质量。在商业上，将该产品添加到现有产品(评估蜥蜴)中，该产品提供针对循证计划评估而量身定做的前测和后测调查，将显著加强我们在预防评估市场的利基市场。这一SBIR项目的广泛目标是开发一种工具，能够收集由于被列入NREPP示范方案清单而被传播的广泛方案以及希望有资格纳入未来的新开发方案的交付质量的标准化衡量标准。在第一阶段，我们采访了预防研究人员、项目管理员、项目开发人员和政策制定者，以定义和阐述定义实施质量维度的基本标准。根据他们的输入，我们设计了一个原型系统，使我们能够完成以下任务：(1)为选定的NREPP辅导员提供的预防计划创建会话和计划评级表单模板，(2)将模板链接到数据库以跟踪表单信息(计划、会话、观察员、教师和创建日期)，以及(3)创建远程打印的PDF表单，可以通过条形码链接回数据库。原型系统测量剂量、依从性、接合度和适应性。我们还开发了一本使用该系统的手册。虽然该系统可用于试点测试，但它还不是完全自动化的，也不包括许多要素。在第二阶段，我们的第一项任务将是完成交付质量数据收集系统，以收集交付质量数据，其中将包括评估遵守、适应、剂量、参与度和总体质量衡量标准的工具。我们将开发在线应用程序，使收集的有关课程交付质量的数据能够以各种方式与学生的测试前和测试后结果数据联系起来并进行联合分析。为了确保应用程序按预期工作，我们将对所有组件应用程序进行阿尔法和贝塔测试。我们将开发一个在线系统，用于进行基本分析和准备报告，以满足五个不同专业群体的需求：计划实施者、计划管理员、计划开发人员、预防研究人员和政策制定者。我们将与10个预防机构和学区进行现场试验，以证明交付质量数据收集和报告系统为项目实施者、预防机构管理员和监督员以及政府机构预防管理员提供有助于记录和了解预防项目交付过程的信息。现有的预防机构将采用评价蜥蜴系统，包括测试前-测试后调查和交付质量报告表格。我们假定，参与团体将报告对该系统的运作感到满意，并将报告对如何在其职权范围内实施预防的认识提高。此外，我们将进行一项信度和效度分研究，要求参与机构选择一名教师对他们教授的课程进行录像，以便将观察数据纳入分析，以证明这些措施的可靠性和有效性。 与公共健康相关：药物滥用预防研究领域已经确定了许多可以有效的干预措施；然而，除非方案在实践中得到很好的实施，否则它们不太可能达到预期的效果，这些方案的努力和费用可能会被浪费。人们普遍认识到需要评估执行质量，但迄今为止，还没有收集或报告有关数据的统一战略。我们提议的产品的价值在于，它将提供一个易于使用的系统，用于记录方案执行的质量，并可与结果数据相联系。

DESCRIPTION (provided by applicant): Quality of delivery is known to moderate the effectiveness of drug abuse prevention programs. As research-based programs become widely disseminated, having tools for assessing quality of delivery becomes important for documenting and understanding the conditions under which programs succeed and fail. There are currently no standard methods for measuring the quality with which prevention programs are delivered. The goal of this project is to create a standardized tool for assessing quality of delivery for research-based drug abuse prevention programs.. Commercially, the addition of this product to an existing product (Evaluation Lizard) that provides pretest and posttest surveys tailored to the evaluation of evidence-based programs, will significantly strengthen our niche in the prevention evaluation marketplace. The broad aim of this SBIR project is to develop a tool that will allow the collection of standardized measures of quality of delivery across a wide range of programs being disseminated as a result of their inclusion on the NREPP model programs list and newly developed programs that wish to qualify for future inclusion. During Phase I, we interviewed prevention researchers, program administrators, program developers, and policy makers to define and elaborate on the essential criteria for defining dimensions of quality of implementation. Based on their input, we designed a prototype system that allowed us to accomplish the following tasks: (1) create session and program rating form templates for selected NREPP facilitator-delivered prevention programs, (2) link templates to a database to track form information (program, session, observer, teacher, and date of creation), and (3) create PDF forms printed remotely that can be linked by bar code back to the database. The prototype system measures dosage, adherence, engagement, and adaptation. We also developed a manual for using the system. While functional for the pilot test, this system is not yet fully automated and does not include numerous elements. During Phase II, our first task will be to complete the quality of delivery data gathering system for collecting quality of delivery data that will include tools for assessing adherence, adaptation, dosage, engagement, and measures of overall quality. We will develop online applications that will allow data collected about quality of program delivery to be linked and jointly analyzed in various ways with pretest-posttest outcome data from students. To ensure that the application works as intended, we will conduct alpha and beta tests of all component applications. We will develop an online system for conducting basic analysis and preparing reports that meets the needs of five separate professional groups: program implementers, program administrators, program developers, prevention researchers, and policy makers. We will conduct a field trial with 10 prevention agencies and school districts to demonstrate that the quality of delivery data collection and reporting system provide program implementers, prevention agency administrators and supervisors, and government agency prevention administrators with information useful to documenting and understanding the process of prevention program delivery. Established prevention agencies will adopt the use of the Evaluation Lizard system, including both pretest-posttest surveys and quality of delivery report forms. We hypothesize that participating groups will report satisfaction with the functioning of the system and will report increased awareness of how prevention is being implemented in their area of purview. Further, we will conduct a reliability and validity sub-study in which participating agencies will be asked to select one teacher to video record the lessons they teach so that observation data can be included in the analysis in order to demonstrate reliability and validity of the measures. PUBLIC HEALTH RELEVANCE: The field of drug abuse prevention research has identified numerous interventions that can be effective; however, unless programs are implemented well in practice, they are unlikely to achieve their desired effect, and the effort and expense of these programs may be wasted. The need to assess quality of implementation is widely recognized, but, to date, there is no unified strategy for gathering or reporting about data. The value of the product we are proposing is that it would provide an easy-to-use system for documenting quality of program implementation that could be linked to outcome data.