Data Analysis Practices in Drug Prevention Evaluation

描述(由申请人提供)：研究表明，对一些最广泛倡导的以学校为基础的酒精和毒品预防计划的评估在数据分析和呈现方面使用了可疑的做法，例如在众多结果变量中的选择性报告、分析前测量尺度的变化、多分组分析以及选择性地使用高于0.05的阿尔法水平和单尾显著性检验(Brown，2001；Gerstein&Green，1993；Gorman，1998；2002a；2002b；2003；in-a；in-Press-b；Manski，Pepper&Petrie，2001；Moskowitz，1989；1993)。这导致高估了这些方案在预防儿童和青少年酗酒和吸毒方面的作用。 然而，目前尚不清楚这种做法在酒精和毒品预防的整个领域中的普遍程度，因为现有的批评焦点一直集中在相当狭窄的学校项目上。对以社区和家庭为基础的干预措施的关注很少，只对最突出的以学校为基础的课程进行了详细审查。因此，这里描述的项目旨在以比迄今进行的更系统和更全面的方式评估酒精和毒品预防方案研究基地的质量。因此，它将处理以下具体问题： I.在已出版的药物和酒精预防研究文献中，不规范和有问题的数据管理和分析做法的流行率是多少？ 二、使用这种不规范和有问题的数据管理和分析实践是否会导致对计划效果的严重高估？ 三、酒精和毒品预防计划中有多大比例是独立评估的(即由项目制定者以外的研究人员进行评估)？ 四、独立进行的评估是否采用了更好的数据管理和分析实践？ 这些问题将通过以下两个程序得到解决：第一，对预防药物滥用中心示范药物滥用方案清单上出现的54个方案的评价中使用的数据管理和分析做法进行系统审查； 第二，重新分析来自一项独立的反药物滥用教育(DARE)评估的数据，该评估结果主要是无效的。

DESCRIPTION (provided by applicant): Research has shown that evaluations of some of the most widely advocated school-based alcohol and drug prevention programs employ questionable practices in their data analysis and presentation such as selective reporting among numerous outcome variables, changes in measurement scales before analysis, multiple subgroup analysis, and selective use of alpha levels above 0.05 and one-tailed significance tests (Brown, 2001; Gerstein & Green, 1993; Gorman, 1998; 2002a; 2002b; 2003; in press-a; in-press-b; Manski, Pepper & Petrie, 2001; Moskowitz, 1989; 1993). This has resulted in an overestimation the effects that these programs have in terms of preventing alcohol and drug use among children and youth. However, it is unclear just how widespread the use of such practices is within the overall field of alcohol and drug prevention since the focus of the existing critiques has been on a fairly narrow range of school-based programs. Very little attention has been focused on community and family-based interventions, and only the most prominent school-based curricula have been examined in any detail. The project described herein is therefore intended to assess the quality of the research base of alcohol and drug prevention programs in a more systematic and comprehensive manner than has been conducted to date. Accordingly, it will address the following specific questions: i. What is the prevalence in the published drug and alcohol prevention research literature of irregular and questionable data management and analysis practices? ii. Does the use of such irregular and questionable data management and analysis practices result in significant overestimation of program effects? iii. What proportion of alcohol and drug prevention programs is independently evaluated (that is, evaluated by researchers other than those who developed the program)? iv. Do evaluations that are conducted independently employ better data management and analysis practices? These questions will be addressed through the following two procedures: First, a systematic review of the data management and analysis practices used in the evaluations of the 54 programs t hat appear on the Center for Substance Abuse Prevention's list of model substance abuse programs; Second, reanalysis of data from an independent Drug Abuse Resistance Education (DARE) evaluation that produced predominantly null results.