Missing Data as Useful Data

及时而安全的决策需要实时摄取和同化与系统动力学相关的数据，认识到不完整性、不确定性、固有的代表性不足/过度代表性和偏见。该奖学金将设计和实施新的程序，以适应“已发现”或“新形式”数据（如社交媒体和移动电话数据）中缺失和偏差的来源和操作，并提出与传统统计来源进行三角测量的新方法。它将提供通用和新颖的方法，以高效、有效和安全的方式使用“新形式的数据”。我们的“数字化”生活和社交媒体的普及、无处不在的传感器和小工具为我们提供了前所未有的机会，以比传统调查和民意调查更高的频率了解社会、经济、福祉和物理世界。然而，这些信息通常是通过不受控制的记录机制获得的，例如，观察性地，在过度/不足的代表性和偏见，缺失，稀疏性和潜在依赖性方面提出挑战。这阻碍了这些数据源与传统数据源的有效集成，从而构建更丰富、更全面的资源，以构建和培训最新的统计和尖端的深度学习人工智能模型。从这些数据中产生的任何决策、模式和模型——即使它们构成了人口的大多数——都可能忽视那些不参与的人的需求。基础统计数据将根据奖学金第一阶段提供的概念验证证据开发，将应用于广泛的应用和学科，包括警务（例如，少报犯罪、隐藏的在线危害）、社会关怀、公共卫生、包容性城市规划，与国家统计局（ONS）战略、动态人口普查保持一致。该奖学金将开发基于范式转换视角的新模型和框架，该视角考虑甚至使用偏差、稀疏性和缺失数据作为有用数据。它将提供解决方案和机制，以可靠地使用“整个数据集”，并整合用户生成的数据和传统的调查数据，以制定有意义、现实和及时的数据驱动政策和决策。新颖性：-考虑将“新形式的数据”作为有用的数据与传统数据进行整合/三角测量，以提供对系统的可靠、及时和最新的理解，可以开辟广泛的具有国家重要性和战略意义的应用，包括管理漏报的犯罪、更好的社会关怀和社会保护、包容性城市规划以及使用行政和替代数据进行动态人口普查。-将缺失的数据视为有用的数据，同时使用可用和不可用的数据来弥补缺失的数据。-提供有效的程序，将新形式的数据与传统数据集结合起来，并提供可量化的质量和适用性措施。-使用新形式数据的伦理、法律和责任方面的考虑，例如我们没有的数据伦理，可以开启关于伦理、法律、安全、公平、可靠性、安全性、透明度和问责制的更广泛讨论。虽然它提高了包容性，使闻所未闻的事情变得更加明显，但有关机构、隐私和数据更广泛利益的道德问题。这项奖学金将支持我建立我的不断壮大的团队和我的研究领域，以提供世界级的基础和应用研究，涉及“新形式的数据”。在此过程中，我提出了一套方法和机制，使非标准数据源（可能与传统数据结合使用）能够有效地利用，以最大限度地提高效益，并提供对城市和社会的（近）实时理解。

Timely yet safe decisions require real-time ingestion and assimilation of data pertaining to system dynamics, cognisant of incompleteness, uncertainty, inherent under/over-representation, and bias. This fellowship will devise and implement novel procedures for accommodating the source and operation of missingness and biases in 'found' or "new forms of" data, such as social media and mobile phone data, and propose new ways of triangulating them with traditional statistical sources. It will provide generic and novel methods to use "new forms of data" in ways that are efficient, effective, and safe to use. Our "digitalised" lives and the popularity of social media, ubiquitous sensors, and gadgets have provided us with an unprecedented opportunity to understand society, the economy, wellbeing, and the physical world at a much higher frequency than traditional surveys and polls. However, this information will normally be obtained with uncontrolled recording mechanisms, e.g. observationally, presenting challenges around over/under-representation and biases, missingness, sparsity, and latent dependencies. This inhibits these sources from being integrated effectively with traditional data sources to build a richer, more comprehensive, resource to build and train the latest statistical and cutting-edge deep learning AI models. Any decisions, patterns, and models that arise from such data-even if they constitute the majority of the population-can overlook the needs of those who do not participate. The foundational statistics that will be developed based on proof-of-concept evidence delivered in the first phase of the fellowship, will be applied to a wide range of applications and disciplines, including policing (e.g. under-reported crime, hidden online harms), social care, public health, inclusive city planning, aligned with Office for National Statistics (ONS) strategy, the dynamic census. This fellowship will develop novel models and frameworks based on a paradigm-shifting perspective that considers or even use biases, sparsity, and missing data as useful data. It will provide solutions and mechanisms to reliably use "whole datasets" and integrate user-generated data and traditional survey data to have meaningful, realistic, and timely data-driven policies and decisions. Novelty: - Considering "new forms of data" as useful data to be integrated/triangulated with traditional data to provide a reliable, timely, and updated understanding of the systems can open up a wide range of applications that are nationally important and strategic, including managing under-reported crime, better social care and protection of society, inclusive city planning, and dynamic census using administrative and alternative data. - Considering missingness as useful data, enabling the use of both available and unavailable data to compensate for the missing data. - Providing an effective procedure to combine new forms of data with traditional datasets with quantifiable measures for quality and fitness for purpose. - Ethical, legal, and liability considerations of using new forms of data, such as ethics of data we do not have, can open a wider discussion about the ethical, legal, security, fairness, reliability, safety, transparency, and accountability. While it improves inclusivity and makes the unheard more visible, the ethical questions regarding agency, privacy and wider benefit of data. This fellowship will support me to establish my growing team and my area of research to deliver world-class fundamental and applied research involving "new forms of data". In doing so, I deliver a suite of methods and mechanisms that enable the effective use of non-standard data sources (potentially in conjunction with traditional data) to maximise benefits and deliver a (near) real-time understanding of cities, and societies.