Alternative Metrics of Impact in Multilateral Agencies: The Potential for Knowledge for Sustainable Human Development

点击翻译按钮获取中文摘要

Globally, over the past decades we have made progress in the eradication of poverty and the reduction of inequalities. However, there are still many global challenges to be addressed. These complex development challenges require open, innovative systems driven by research and evidence. To have impact, this research must must be taken up, not only by researchers, but also by policymakers and practitioners. In their pursuit to produce relevant, useful research towards sustainable development, research producers in the field are invested in understanding how their work is used by diverse audiences. As such, they are eager to develop efficient ways to measure wider societal impact, in order to inform their strategies around research focus, funding, communication and reporting.Currently development research institutions use a range of tools to measure the importance and influence of the research they produce. For example, expert peer review can be used to assess quality, quantitative metrics can hint at influence, and case studies can indicate effects on policy. However, measuring wider impact on policymakers and practitioners is difficult and often slow and inefficient, and each of these tools have strengths and limitations. Against the backdrop of the digital economy and big data, alternative metrics of impact have grown rapidly as a way of measuring impact beyond academia. 'Altmetrics' are tools designed to gauge the impact of diverse research outputs on diverse audiences. Capitalising on the huge growth of social media and social networking, altmetrics provide new data: (a) mentions in policy documents, blogs, news, Wikipedia, (b) social media 'shares' (e.g. Twitter, Facebook), and (c) social 'bookmarks' (e.g. Mendeley, ResearchGate). These tools have grown rapidly in the last five years, and have now been taken up by influential universities and publishers. Thus, it is crucial to examine their growing role in the current research context.This project will investigate the effects of altmetrics for multilateral aid agencies. Multilaterals are dominant research centres, but the production, dissemination and uptake of their research are poorly understood. Strongly invested in understanding how their work is utilised by users around the world, these agencies have begun to gather evidence from altmetric sources. Yet, the value of these tools is unclear and often controversial, despite their potential to provide more nuanced and timely understanding of impact in policy and practice contexts. The multilateral agencies I will work with on this project are engaged with altmetrics to varying degrees, but remain stalled in internal debate about their use.As the first sociological study on the effects of altmetrics for knowledge production in institutions, the project will generate a wealth of new qualitative and quantitative data through an innovative methodological approach. The project seeks to: (i) determine the nature and effects of altmetrics in multilateral agencies, (ii) inform the goals and strategies of research and policy institutes, and (iii) expand knowledge in sociology, research evaluation, research policy, and development studies. This project will meet its aims through (i) two comparative ethnographic case studies, (ii) network analysis using altmetric data, and (iii) supplementary interviews with research producers and users. This novel methodological combination will allow me to chart the networks of research impact, but also, crucially, to capture the complexity of meanings and practices surrounding altmetric use. Using a novel sociological, mixed-methods framework, this project will provide new theoretical and empirical insights to understand the use of alternative metrics of impact in the global policy context. These contributions will be delivered through a number of academic and user-focused outputs, towards better utilisation of knowledge and evidence in addressing global challenges.