Doctoral Dissertation Research: A Comparative Evaluation of International Models for Evaluating Federally-Funded Research

博士论文研究：评估联邦资助研究的国际模型的比较评估

• 批准号: 0721097
• 负责人: Michael Scriven
• 金额: $ 0.2万
• 依托单位: Western Michigan University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2007
• 资助国家: 美国
• 起止时间: 2007-06-01 至 2008-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0721097&HistoricalAwards=false
• 关键词: Doctoral; Dissertation; Research; Comparative; Evaluation

In the last few decades the evaluation of research has become a high-stakes enterprise. With increasing political governance and federal budgets often in the billions, the livelihood of individual researchers, research groups, departments, programs, and entire institutions often swing in the balance. Simultaneously, many of the longstanding principles and practices often lead to poor decisions about the actual or prospective merits of researchers and their research.This dissertation research describes, classifies, and comparatively evaluates national models used to evaluate research and allocate research funding in sixteen countries. These models vary widely in terms of how research is evaluated and financed, but nearly all share the common characteristic of relating funding to past performance. Independent, blinded panels of researchers and evaluators in two countries rated sixteen national models on more than 25 quality indicators. They then ranked the national models in terms of their validity, credibility, utility, cost-effectiveness, and ethicality. The results of the rankings show that the clear leaders are nations using large-scale research assessment exercises of various hues. Bulk funding and indicator-driven models received substantially lower ratings and thus ranked far lower in comparison to the assessment exercises. The project concludes with suggested implications for research evaluation practice and policy.

在过去的几十年里，对研究的评估已经成为一项高风险的事业。随着政治治理的增加和联邦预算的增加，个人研究人员、研究小组、部门、项目和整个机构的生计往往摇摆不定。同时，许多长期存在的原则和实践往往导致对研究人员及其研究的实际或预期价值的糟糕决策。本论文研究描述、分类和比较评估了16个国家用于评估研究和分配研究资金的国家模型。在如何评估和资助研究方面，这些模型差别很大，但几乎所有模型都有一个共同的特点，即将资金与过去的业绩联系起来。两个国家的独立、盲目的研究人员和评估员小组在25个以上的质量指标上对16个国家模型进行了评级。然后，他们根据有效性、可信度、实用性、成本效益和伦理道德对国家模式进行了排名。排名结果显示，明显领先的国家是使用各种颜色的大规模研究评估练习的国家。大宗供资和指标驱动型模式的评级要低得多，因此与评估工作相比排名要低得多。该项目最后提出了对研究评估实践和政策的建议含义。