The reliability of evidence review methodology in environmental science and conservation
The reliability of evidence review methodology in environmental science and conservation
复制标题
环境科学与保护中证据审查方法的可靠性
DOI：
10.1016/j.envsci.2016.06.012
复制
发表时间：
2016
影响因子：
6
通讯作者：
O'Leary B
中科院分区：
文献类型：
--
作者：
O'Leary B
鉴于主要研究文章的激增，可靠的环境证据审查对政策和管理决策的重要性正在增加。虽然进行审查是综合分散的主要证据基础的有效方法，但方法可靠性差的审查有可能因未准确反映现有证据基础而造成误导。为了评估证据审查对决策的当前价值，我们使用环境证据综合评估工具（CEESAT）评估了2015年初发表的文章的系统样本（N = 92）。CEESAT根据客观性、透明度和全面性方面的重要因素，评估与政策相关的证据审查方法。总体而言，评论表现不佳，中位数得分为2.5/39，众数得分为零（范围0-30，平均5.8），低分在各学科领域普遍存在。总体而言，采用荟萃分析技术的综述得分高于叙述性综合（中位数分别为18.3和2.0），因为后者始终未能充分报告方法或如何得出结论。然而，一些叙述性综合获得了高分，说明审查的可靠性应逐案评估。鉴于审查对于为管理和政策以及研究提供信息具有潜在的重要性，因此必须提高总体方法的可靠性。尽管越来越多的系统性综述和荟萃分析强调了正在取得的一些进展，但我们的研究结果表明，在过去十年中几乎没有改善。为了推动进展，我们建议每年对证据审查的方法可靠性进行评估。为了更好地服务于环境政策和管理界，我们确定了对审查方法进行独立批判性评估的要求，从而使决策者能够选择最有可能准确反映证据基础的审查。
Given the proliferation of primary research articles, the importance of reliable environmental evidence reviews for informing policy and management decisions is increasing. Although conducting reviews is an efficient method of synthesising the fragmented primary evidence base, reviews that are of poor methodological reliability have the potential to misinform by not accurately reflecting the available evidence base. To assess the current value of evidence reviews for decision-making we appraised a systematic sample of articles published in early 2015 (N = 92) using the Collaboration for Environmental Evidence Synthesis Assessment Tool (CEESAT). CEESAT assesses the methodology of policy-relevant evidence reviews according to elements important for objectivity, transparency and comprehensiveness. Overall, reviews performed poorly with a median score of 2.5/39 and a modal score of zero (range 0–30, mean 5.8), and low scores were ubiquitous across subject areas. In general, reviews that applied meta-analytical techniques achieved higher scores than narrative syntheses (median 18.3 and 2.0 respectively), as a result of the latter consistently failing to adequately report methodology or how conclusions were drawn. However, some narrative syntheses achieved high scores, illustrating that the reliability of reviews should be assessed on a case-by-case basis. Given the potential importance of reviews for informing management and policy, as well as research, it is vital that overall methodological reliability is improved. Although the increasing number of systematic reviews and meta-analyses highlight that some progress is being made, our findings suggest little or no improvement in the last decade. To motivate progress, we recommend that an annual assessment of the methodological reliability of evidence reviews be conducted. To better serve the environmental policy and management communities we identify a requirement for independent critical appraisal of review methodology thus enabling decision-makers to select reviews that are most likely to accurately reflect the evidence base.
登录
查看更多内容
应对海洋保护区荟萃分析中的发表偏差
DOI：
--
发表时间：
2011
期刊：
影响因子：
--
作者：
B. Huntington
通讯作者：
B. Huntington
DOI：
10.1177/0002716203254762
发表时间：
2003-09-01
影响因子：
2.8
作者：
Chalmers, I
通讯作者：
Chalmers, I
影响因子：
3.3
作者：
A. Pullin;T. Knight
通讯作者：
T. Knight
评论文章是支持保护和环境管理的可靠证据来源吗？
影响因子：
5.9
作者：
P. Roberts;G. Stewart;A. Pullin
通讯作者：
A. Pullin
哪门科学 - 谁的科学 - 科学学科如何塑造环境法
DOI：
--
发表时间：
2011
期刊：
影响因子：
--
作者：
Eric Biber
通讯作者：
Eric Biber