Record Linkage Across Heterogeneous Data Sources

点击翻译按钮获取中文摘要

In the context of creating longitudinal data from census data, record linkage refers to finding the same person across several censuses. The recent emergence of 100 percent national census collections enables a systematic identification and linking of the same individuals across censuses in order to create a new database of individual life-course information. The main challenge is that unique identifiers do not exist for this historical data, thus one must use attributes common to all of the databases and compare their values to determine whether two records refer to the same entity. Other challenges are presented by different database formats, typographical errors, missing data and ill-reported data (both intentional and inadvertent). Furthermore, not everyone in a census is present in the next one because death and emigration remove people from the population, while births and immigration add new people who were not present in the previous census but who may have characteristics similar to those who were present. Finally, processing the cross product of millions of records when linking two census collections presents significant computation challenges. **The overall objective of my research is to create and extract knowledge from historical longitudinal data. The motivation driving my research is to significantly advance the understanding of the Canadian society through computational science. As part of this proposal, I will focus on two goals: the short term goal is the creation of large scale longitudinal data from historical censuses, a complex yet critical step towards my objective; and the long term goal of automatically extracting useful knowledge from the longitudinal data that would enrich our understanding about the history and economics of the Canadian society.**The recent emergence of 100 percent digitized Canadian census collections enables for the first time a large scale, data-driven, understanding of key society changes such as migration, social mobility, labour market adjustments and intergenerational inequality. The main challenge from a social science perspective is the large scale generation of individual life-course information due to the mostly manual linking techniques employed, strongly limiting the data available for their studies. A first key impact*of my research will consist of automatic large scale generation of longitudinal data from historical censuses. To achieve this, I will significantly advance the state-of-the art in the automated record linkage through the five key results that will act as milestone towards my short term goal: better feature construction; tighter bounds for*candidate selection; more accurate classification models; increased linking coverage through the use of family information; and, a standardized benchmark for evaluating and validating historical record linkage. I will share the resulting longitudinal data with researchers in history and the social sciences; they have been waiting for longitudinal data of this nature and scale in order to resolve pressing research questions about society, history and economy. A second key impact will consist of employing knowledge extraction techniques on the generated longitudinal data to identify interesting patterns about the Canadian society. I expect that this result will also require devising new knowledge extraction techniques that are better at identifying patterns in large-scale longitudinal data, which should also make them suitable for related research areas such as social networks. Moreover, by sharing the patterns with social researchers, we will both validate and further advance the understanding of the socio-economic*changes of the Canadian society.