Using multi-level multi-source auxiliary data to investigate nonresponse bias in UK general social surveys

点击翻译按钮获取中文摘要

This project will explore the extent to which the predictive power of various forms of "Big Data" can be harnessed to overcome the impact of poor response to surveys - one of the major challenges facing social research today. Social surveys are a key tool used by the media, policy makers, and academics to understand more about public attitudes and behaviour. However, the value of surveys is put at risk by the fact that a large and growing number of those selected to take part in surveys do not respond. As non-respondents may be very different from respondents, nonresponse can introduce significant bias into the conclusions drawn from survey data. There is a pressing need therefore to understand more about the extent and sources of nonresponse bias. This requires having information about both respondents and non-respondents. In the absence of interview data being available for non-respondents, this information must be obtained from other, external, sources. The growth in "Big Data" i.e. routinely generated data arising from commercial transactions, online communication or public administration provides exciting new opportunities to supplement survey data with data from other sources. As opportunities for data linkage increase, there is a need for a detailed investigation into how such data can be used to understand and hopefully correct for nonresponse bias in general social surveys. This project will conduct such an investigation by adding pre-existing data from multiple sources to UK data from the European Social Survey (ESS), a methodologically rigorous survey of public attitudes. The project, drawing on the expertise of an inter-disciplinary team of survey researchers, statisticians and geographic information (GI) specialists, has three strands: First, the project will explore the opportunities that exist for matching data from three different sources to survey data. These include: small-area administrative data; commercial marketing data and geocoded information from the Ordnance Survey. Each data source will be evaluated in terms of: what information it can provide which may be matched to the survey records of respondents and non-respondents; the accuracy and completeness of this information; and the challenges that matching data presents in terms of the increased risk of individuals or households being identified from the combination of data held about them. Second, we will see how the matched data can provide information about potential biases that may be present in the survey data as a result of nonresponse. This will involve identifying any external variables associated both with the likelihood of nonresponse and the attitudes and behaviour the survey intendeds to measure. The project will consider how sources of nonresponse bias may vary geographically across the UK. Finally, we will assess whether using these external variables to create nonresponse weights to adjust for the possible over or under representation of certain types of respondent in the dataset has a significant effect on survey estimates and reduces bias in the data. This project has the potential to contribute significantly to our understanding not only of survey nonresponse bias but also the statistical tools available to remedy this bias, to improve survey data collection and generate more robust data to better understand public attitudes and behaviour. Lessons learnt will enhance general social surveys in the UK and internationally. This will have considerable benefits for the wide range of stakeholders involved in the funding, collection, and analysis of survey data and those who rely on the insights it provides. This includes academics, government agencies and other publically funded bodies, third sector organisations, policy makers and, ultimately, the general public.