Advanced Theory and Methods for Evaluating the Utility and Privacy Risks of Synthetic Health Data

评估综合健康数据的实用性和隐私风险的先进理论和方法

• 批准号: RGPIN-2022-04811
• 负责人: ElEmam, Khaled
• 金额: $ 1.75万
• 依托单位: University of Ottawa
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=753398
• 关键词: Advanced; Theory; Methods; Evaluating; Utility

Access to health data for secondary purposes remains a challenge because of privacy concerns. Synthetic data generation (SDG) has been proposed to enable data sharing that is believed to have low identification risks because there is no one-to-one mapping to real individuals. However, if the generative models used to generate synthetic data are overfit, or if a dataset is categorical with a small number of possible combinations of values, then real records may be generated. The adoption of SDG will also depend on demonstrating the utility of the generated data. Utility is broadly defined as the ability to replicate the conclusions from the analysis of real data on synthetic data. SDG needs to simultaneously optimize on privacy and utility. However, thus far SDG loss functions have largely been focused on maximizing utility, and privacy risks are often assessed after the data are generated. The purpose of this program is to develop a unified privacy framework for SDG, and to evaluate and improve current utility metrics. These results would then be used to define and test a combined loss metric that can be applied to optimize the generation of synthetic data which allows for the simultaneous management of privacy and utility. Privacy Evaluation Our focus in this program will be on identity disclosure conditional on attribute disclosure and membership disclosure. We will develop and validate a unified risk model that integrates identity, attribute, and membership disclosure. Currently there are no privacy models that are directly applicable to longitudinal synthetic datasets. The unified model of disclosure above will be extended to longitudinal data with multiple heterogeneous events per patient. Existing approaches used in the disclosure control literature will be incorporated into the synthetic data privacy model. Utility Evaluation Utility metrics can serve multiple purposes such as model optimization and synthetic dataset evaluation to accept or reject specific generated datasets. In this part of the program, current utility metrics will be empirically evaluated. The results will clarify which utility metrics are useful for optimization, and synthesized dataset acceptance/rejection. Currently, there has been a dearth of work on evaluating the utility of synthetic longitudinal data. Simple approaches such as concordance between k-order Markov chains capture some structural properties, but do not provide measures related to analytic workloads. This program of research will extend and evaluate the utility metrics for longitudinal data. Risk-Utility Optimization With appropriately defined privacy and utility metrics, a combined risk-utility measure can be defined and used as an optimization criterion for SDG algorithms. This will ensure that generated synthetic data satisfy both criteria by construction. Such a measure will be evaluated on common SDG algorithms used on health data.

出于隐私方面的考虑，为次要目的获取卫生数据仍然是一个挑战。合成数据生成（SDG）被提出用于实现数据共享，这种共享被认为具有较低的识别风险，因为没有与真实个体的一对一映射。但是，如果用于生成合成数据的生成模型是过拟合的，或者如果数据集具有少量可能的值组合的分类，则可能生成真实记录。可持续发展目标的采用还将取决于所生成数据的实用性。效用的广义定义是将真实数据分析得出的结论复制到合成数据上的能力。可持续发展目标需要同时优化隐私和效用。然而，到目前为止，可持续发展目标损失函数主要集中在效用最大化上，隐私风险通常是在数据生成后评估的。该计划的目的是为可持续发展目标制定统一的隐私框架，并评估和改进当前的效用指标。然后，这些结果将用于定义和测试一个综合损失度量，该度量可用于优化合成数据的生成，从而允许同时管理隐私和效用。隐私评估我们在这个项目中的重点是身份披露，前提是属性披露和成员披露。我们将开发并验证集成身份、属性和成员披露的统一风险模型。目前还没有直接适用于纵向合成数据集的隐私模型。上述公开的统一模型将扩展到每个患者具有多个异构事件的纵向数据。在披露控制文献中使用的现有方法将被纳入综合数据隐私模型。效用评估效用指标可以用于多种目的，例如模型优化和综合数据集评估，以接受或拒绝特定生成的数据集。在程序的这一部分中，将对当前的效用度量进行经验评估。结果将澄清哪些效用指标对优化有用，以及合成数据集的接受/拒绝。目前，在评价综合纵向数据的效用方面缺乏工作。简单的方法，如k阶马尔可夫链之间的一致性，可以捕获一些结构属性，但不提供与分析工作负载相关的度量。这个研究项目将扩展和评估纵向数据的效用度量。通过适当定义隐私和效用指标，可以定义组合的风险效用度量，并将其用作可持续发展目标算法的优化标准。这将确保生成的合成数据通过构造满足这两个标准。将根据用于卫生数据的共同可持续发展目标算法对这一措施进行评估。