Doctoral Dissertation Research: A Comparative Evaluation of International Models for Evaluating Federally-Funded Research

在过去的几十年里，研究评估已经成为一项高风险的事业。随着政治治理和联邦预算的增加，个体研究人员、研究小组、部门、项目和整个机构的生计经常在平衡中摇摆。同时，许多长期存在的原则和实践常常导致对研究人员及其研究的实际或预期价值的错误决定。本文对16个国家用于评估研究和分配研究经费的国家模型进行了描述、分类和比较评价。这些模式在如何评估和资助研究方面差异很大，但几乎所有模式都有一个共同的特点，即将资助与过去的表现联系起来。两个国家的独立、盲法研究人员和评估人员小组根据超过25项质量指标对16个国家的模型进行了评级。然后，他们根据有效性、可信度、效用、成本效益和道德标准对国家模式进行排名。排名结果显示，明显领先的是那些使用各种各样的大规模研究评估活动的国家。大量供资和指标驱动的模式得到的评级要低得多，因此与评估活动相比排名要低得多。该项目最后提出了对研究评估实践和政策的建议。

In the last few decades the evaluation of research has become a high-stakes enterprise. With increasing political governance and federal budgets often in the billions, the livelihood of individual researchers, research groups, departments, programs, and entire institutions often swing in the balance. Simultaneously, many of the longstanding principles and practices often lead to poor decisions about the actual or prospective merits of researchers and their research.This dissertation research describes, classifies, and comparatively evaluates national models used to evaluate research and allocate research funding in sixteen countries. These models vary widely in terms of how research is evaluated and financed, but nearly all share the common characteristic of relating funding to past performance. Independent, blinded panels of researchers and evaluators in two countries rated sixteen national models on more than 25 quality indicators. They then ranked the national models in terms of their validity, credibility, utility, cost-effectiveness, and ethicality. The results of the rankings show that the clear leaders are nations using large-scale research assessment exercises of various hues. Bulk funding and indicator-driven models received substantially lower ratings and thus ranked far lower in comparison to the assessment exercises. The project concludes with suggested implications for research evaluation practice and policy.