Models and applications for measuring the impact of health research: update of a systematic review for the Health Technology Assessment programme
Models and applications for measuring the impact of health research: update of a systematic review for the Health Technology Assessment programme
复制标题
DOI：
10.3310/hta20760
复制
发表时间：
2016-10-01
影响因子：
3.6
通讯作者：
Blatch-Jones, Amanda
中科院分区：
文献类型：
--
作者：
Raftery, James;Hanney, Steve;Blatch-Jones, Amanda
背景资料：本报告在2007年综述的基础上，对衡量研究项目影响的方法和工具进行了综述，并对其进行了扩展，目的是：（1）确定衡量卫生研究项目影响的理论模型和经验方法的范围;（2）对模型和方法进行分类;（3）总结这些模型的应用和使用证据;（4）确定卫生研究项目影响的评估方法。资料来源：检索2005 - 01/2014 - 08奥维德MEDLINE、EMBASE、Cumulative Index to Nursing and Allied Health Literature和The科克伦图书馆。2014年8月至2015年3月，我们系统检索了8个数据库，并辅以个人知识。经过调整的回报框架仍然是最广泛使用的方法。它借鉴了不同的哲学传统，增强了一个基本的逻辑模型，具有解释性的案例研究元素和对上下文的关注。除了逻辑模式，其他理想类型的方法包括建构主义，现实主义，批判和表演。实践中的大多数模型都实用地借鉴了几种理想类型的元素。影响货币化是一种越来越受欢迎的方法，显示出研究的高回报，但在很大程度上依赖于对健康收益依赖研究程度的假设。尽管在资助试验之前通常需要进行系统评价，但HTA项目并不定期检查这些试验对随后的系统评价的影响。约克/以患者为中心的结果研究所和建议评估、发展和评价工具包的分级提供了评估这种影响的方法，但需要进行评估。本文回顾的文献中，很少有随机试验在阻止使用新技术方面发挥重要作用的例子。少数试验资助的卫生技术评估计划，可能发挥了这样的作用是outlier.Discussion：本次审查的结果支持继续使用的回报框架的卫生技术评估计划。NHS结构的变化，NHS英格兰的发展和国家健康与护理卓越研究所职权范围的变化，为确定和满足当前和未来的研究需求提出了新的挑战。未来对卫生技术评估计划影响的评估将不得不考虑更广泛的变化，特别是评估大学研究质量的卓越研究框架（REF）似乎可能继续依赖案例研究来衡量影响。卫生技术评估方案应考虑如何改进个案研究的格式和选择，以帮助进行更系统的评估。案例研究的选择，如参考框架，但更普遍的是，往往偏向于高影响，而不是低影响的故事。其他行业的经验表明，可以从后者学到很多东西。联合王国大多数主要研究资助者采用researchfish（R）（researchfish有限公司，英国剑桥）对今后的影响评估具有影响。虽然常规收集索引研究出版物有其优点，但researchfish在收集其他非索引产出和活动方面的成功程度仍有待确定。局限性：当我们将重点扩展到2007年审查之外，远远超出仅关注卫生技术评估计划的狭隘范围时，我们在多大程度上能够应对所面临的挑战方面存在局限性。结论：研究资助者可以从继续监测和评估他们资助的研究的影响中受益，他们还应该审查案例研究的贡献，并扩大将试验与荟萃分析和指南联系起来的工作。
Background: This report reviews approaches and tools for measuring the impact of research programmes, building on, and extending, a 2007 review.Objectives: (1) To identify the range of theoretical models and empirical approaches for measuring the impact of health research programmes; (2) to develop a taxonomy of models and approaches; (3) to summarise the evidence on the application and use of these models; and (4) to evaluate the different options for the Health Technology Assessment (HTA) programme.Data sources: We searched databases including Ovid MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature and The Cochrane Library from January 2005 to August 2014.Review methods: This narrative systematic literature review comprised an update, extension and analysis/discussion. We systematically searched eight databases, supplemented by personal knowledge, in August 2014 through to March 2015.Results: The literature on impact assessment has much expanded. The Payback Framework, with adaptations, remains the most widely used approach. It draws on different philosophical traditions, enhancing an underlying logic model with an interpretative case study element and attention to context. Besides the logic model, other ideal type approaches included constructionist, realist, critical and performative. Most models in practice drew pragmatically on elements of several ideal types. Monetisation of impact, an increasingly popular approach, shows a high return from research but relies heavily on assumptions about the extent to which health gains depend on research. Despite usually requiring systematic reviews before funding trials, the HTA programme does not routinely examine the impact of those trials on subsequent systematic reviews. The York/Patient-Centered Outcomes Research Institute and the Grading of Recommendations Assessment, Development and Evaluation toolkits provide ways of assessing such impact, but need to be evaluated. The literature, as reviewed here, provides very few instances of a randomised trial playing a major role in stopping the use of a new technology. The few trials funded by the HTA programme that may have played such a role were outliers.Discussion: The findings of this review support the continued use of the Payback Framework by the HTA programme. Changes in the structure of the NHS, the development of NHS England and changes in the National Institute for Health and Care Excellence's remit pose new challenges for identifying and meeting current and future research needs. Future assessments of the impact of the HTA programme will have to take account of wider changes, especially as the Research Excellence Framework (REF), which assesses the quality of universities' research, seems likely to continue to rely on case studies to measure impact. The HTA programme should consider how the format and selection of case studies might be improved to aid more systematic assessment. The selection of case studies, such as in the REF, but also more generally, tends to be biased towards high-impact rather than low-impact stories. Experience for other industries indicate that much can be learnt from the latter. The adoption of researchfish (R) (researchfish Ltd, Cambridge, UK) by most major UK research funders has implications for future assessments of impact. Although the routine capture of indexed research publications has merit, the degree to which researchfish will succeed in collecting other, non-indexed outputs and activities remains to be established.Limitations: There were limitations in how far we could address challenges that faced us as we extended the focus beyond that of the 2007 review, and well beyond a narrow focus just on the HTA programme.Conclusions: Research funders can benefit from continuing to monitor and evaluate the impacts of the studies they fund. They should also review the contribution of case studies and expand work on linking trials to meta-analyses and to guidelines.