Dependence Modelling with Vine Copulas for the Integration of Unstructured and Structured Data

该项目将开发一种以前从未考虑过的统计数据综合方法，该方法利用多种信息来源提供比目前可用的更准确的预测。今天，我们生活在大数据时代，传统格式的海量数据由公司和组织产生，而社交媒体每秒都会产生大量信息，其中大部分是非结构化的。然而，我们是否有效和高效地利用了官方和社交媒体来源提供的所有信息？这个问题的答案肯定是否定的。大多数用于解决现实世界问题的统计方法都是基于单一信息源的，尽管存在试图利用社交媒体数据的初步工作，但目前还没有能够充分利用非结构化信息及其与其他可用结构化数据的关联的全面和实用的方法。其后果是，非结构化在线数据中包含的宝贵信息继续被忽视和丢失。虽然技术和数字化的进步正在塑造世界，但统计正在努力跟上步伐，目前迫切需要改革其方法和做法。这项提议旨在填补这一空白，充分利用不同信息来源的力量，如传统数据和在线生成的数据，采用开创性和变革性的统计数据整合方法。该项目将支持使用一种新的方法整合非结构化数据和结构化数据的早期研究，这种新方法将构成未来分析的基础，这将导致当前数据方法的根本转变，推动统计走向未来时代。这项研究尚处于早期阶段，但将立即带来有用的结果，该方法将应用于英国西南部地区的犯罪数据，整合我们的项目合作伙伴德文郡和康沃尔警察局(DCP)提供的官方警方信息，以及在不同社交媒体平台上讨论的犯罪数据。我们的办法将对西南部特定地点的犯罪数量和严重程度提供更全面和现实的评估，因为它还将说明未向警方报告但通过社交媒体出现的隐藏犯罪。新闻部将利用这一项目的成果，更有效地规划和组织其干预措施，并在目标领域有效地分配资源。该项目将更深入、更准确地了解包括未报告犯罪在内的刑事犯罪的地理位置，帮助警方更好地支持犯罪高危地区的社区，并及时进行干预，使人们感到更有保障和更安全。这将促进社会包容和更公平的社区，特别是在主要受高犯罪率影响的弱势地区，包括不通过传统渠道举报的犯罪。该项目最初以英国西南部为目标，将为未来的赠款申请奠定基础，以扩大国家一级正在评估的地理区域。此外，由于我们的方法论有无数可能的应用，这个项目将是一个里程碑，将在任何其他科学领域产生进一步的突破，在这些领域中，有多种数据源可用，需要准确的预测。这个项目是及时的，因为它满足了充分利用现有但尚未利用的社交媒体信息的迫切需要。这项研究将为英国在知识提取方面取得领先的国际地位提供一个关键机会，从而带来巨大的社会和经济效益。

The project will develop a statistical data integration methodology, never considered before, that utilizes multiple sources of information to provide more accurate predictions than those currently available. Today we are living in the Big Data era, where masses of data in traditional formats are produced by companies and organizations and large quantities of information, mostly unstructured, are generated by social media, every second. However, are we effectively and efficiently exploiting all the information available to us from official and social media sources? The answer to this question is definitely, no. Most of the statistical approaches used to solve real-world problems are based on a single source of information and, although preliminary work attempting to leverage social media data exists, there are currently no comprehensive and functional methodologies able to fully capitalize on unstructured information and its associations with other available structured data. The consequence is that precious information contained in unstructured online data continues to be neglected and lost. While technology and digitalization advances are shaping the world, statistics is struggling to keep pace and it is currently in critical and urgent need of revolutionizing its methods and practices. This proposal aims at filling this gap, giving life to a pioneering and transformative statistical data integration methodology, fully leveraging the power of different sources of information, such as traditional and online-generated data. The project will support early-stage research on integrating unstructured and structured data using a new methodology based on vine copulas that will form the basis of future analyses, which will lead to a radical transformation of current data approaches, propelling statistics towards the future era. For this research, which is early-stage, yet will bring immediately usable results, the methodology will be applied to data of crimes committed in the South West region of the UK, integrating official police information, provided by our project partner Devon and Cornwall Police (DCP), with crime data discussed on different social media platforms. Our approach will provide a more thorough and realistic appraisal of the volume and severity of crimes in specific locations of the South West, since it will also account for hidden crimes, unreported to the police, but emerging from social media. The results of this project will be used by DCP to more effectively plan and organize their interventions and to efficiently allocate resources in targeted areas. Providing a deeper and more accurate knowledge of the geographical locations of criminal offences, including unreported crimes, this project will assist the police to better support communities in high criminal risk areas with timely interventions, making people feel more protected and safer. This will promote social inclusion and more equitable communities, especially in disadvantaged areas that are mostly affected by high criminality levels, including crimes which are not reported via traditional channels. This project, initially targeting the South West of the UK, will lay the foundation for future grant applications extending the geographical area under assessment at national level. In addition, due to the endless number of possible applications of our methodology, this project will be the milestone that will generate further breakthroughs in any other area of science where multiple data sources are available and accurate predictions are needed. This project is timely since it addresses the urgent need to fully leverage the social media information currently available, but not taken advantage of. This research will provide a key opportunity for the UK to secure a leading international position at the forefront of advances in knowledge extraction, leading to huge social and economic benefits.