Advanced Theory and Methods for Evaluating the Utility and Privacy Risks of Synthetic Health Data

出于隐私考虑，出于次要目的获取健康数据仍然是一项挑战。合成数据生成(SDG)已被提出，以实现被认为具有低识别风险的数据共享，因为不存在与真实个人的一对一映射。然而，如果用于生成合成数据的生成模型过度匹配，或者如果数据集具有少量可能的值组合，则可能会生成真实的记录。SDG的采用还将取决于展示所生成数据的效用。效用的广义定义是在合成数据上复制真实数据分析得出的结论的能力。SDG需要同时优化隐私和效用。然而，到目前为止，SDG损失函数主要集中在最大化效用上，隐私风险通常在数据生成后进行评估。该计划的目的是为SDG开发一个统一的隐私框架，并评估和改进当前的效用指标。然后，这些结果将被用来定义和测试可应用于优化合成数据生成的组合损失度量，从而允许同时管理隐私和效用。隐私评估我们在这个项目中的重点将是身份披露，条件是属性披露和成员披露。我们将开发和验证集成身份、属性和成员披露的统一风险模型。目前还没有直接适用于纵向合成数据集的隐私模型。上述披露的统一模型将扩展到每个患者具有多个不同事件的纵向数据。披露控制文献中使用的现有方法将被纳入合成数据隐私模型。效用评估效用度量可以用于多种目的，例如模型优化和合成数据集评估，以接受或拒绝特定生成的数据集。在该计划的这一部分，将对当前的效用指标进行经验性评估。结果将阐明哪些效用指标对优化有用，以及合成数据集的接受/拒绝。目前，对合成纵向数据的实用性进行评估的工作一直很少。简单的方法，如k阶马尔可夫链之间的一致性，可以捕获一些结构属性，但不提供与分析工作负载相关的度量。这项研究计划将扩展和评估纵向数据的效用度量。风险效用优化通过适当定义隐私和效用度量，可以定义组合的风险效用度量，并将其用作SDG算法的优化标准。这将确保生成的合成数据通过构造满足这两个标准。这一措施将在用于健康数据的常见SDG算法上进行评估。

Access to health data for secondary purposes remains a challenge because of privacy concerns. Synthetic data generation (SDG) has been proposed to enable data sharing that is believed to have low identification risks because there is no one-to-one mapping to real individuals. However, if the generative models used to generate synthetic data are overfit, or if a dataset is categorical with a small number of possible combinations of values, then real records may be generated. The adoption of SDG will also depend on demonstrating the utility of the generated data. Utility is broadly defined as the ability to replicate the conclusions from the analysis of real data on synthetic data. SDG needs to simultaneously optimize on privacy and utility. However, thus far SDG loss functions have largely been focused on maximizing utility, and privacy risks are often assessed after the data are generated. The purpose of this program is to develop a unified privacy framework for SDG, and to evaluate and improve current utility metrics. These results would then be used to define and test a combined loss metric that can be applied to optimize the generation of synthetic data which allows for the simultaneous management of privacy and utility. Privacy Evaluation Our focus in this program will be on identity disclosure conditional on attribute disclosure and membership disclosure. We will develop and validate a unified risk model that integrates identity, attribute, and membership disclosure. Currently there are no privacy models that are directly applicable to longitudinal synthetic datasets. The unified model of disclosure above will be extended to longitudinal data with multiple heterogeneous events per patient. Existing approaches used in the disclosure control literature will be incorporated into the synthetic data privacy model. Utility Evaluation Utility metrics can serve multiple purposes such as model optimization and synthetic dataset evaluation to accept or reject specific generated datasets. In this part of the program, current utility metrics will be empirically evaluated. The results will clarify which utility metrics are useful for optimization, and synthesized dataset acceptance/rejection. Currently, there has been a dearth of work on evaluating the utility of synthetic longitudinal data. Simple approaches such as concordance between k-order Markov chains capture some structural properties, but do not provide measures related to analytic workloads. This program of research will extend and evaluate the utility metrics for longitudinal data. Risk-Utility Optimization With appropriately defined privacy and utility metrics, a combined risk-utility measure can be defined and used as an optimization criterion for SDG algorithms. This will ensure that generated synthetic data satisfy both criteria by construction. Such a measure will be evaluated on common SDG algorithms used on health data.