Advanced Theory and Methods for Evaluating the Utility and Privacy Risks of Synthetic Health Data

由于隐私问题，为次要目的获取健康数据仍然是一个挑战。合成数据生成（SDG）已经被提出来实现被认为具有低识别风险的数据共享，因为不存在到真实的个体的一对一映射。然而，如果用于生成合成数据的生成模型是过拟合的，或者如果数据集是具有少量可能的值组合的分类的，则可以生成真实的记录。 可持续发展目标的采用还将取决于所生成数据的效用。效用被广泛地定义为在综合数据上复制从真实的数据分析得出的结论的能力。 SDG需要同时优化隐私和实用性。然而，到目前为止，SDG损失函数主要集中在效用最大化上，隐私风险通常在数据生成后进行评估。 该计划的目的是为SDG开发一个统一的隐私框架，并评估和改进当前的效用指标。然后，这些结果将用于定义和测试组合损失度量，该组合损失度量可用于优化合成数据的生成，从而允许同时管理隐私和效用。 隐私评估我们在此计划中的重点将是在属性披露和成员披露的条件下的身份披露。我们将开发和验证一个统一的风险模型，该模型集成了身份、属性和成员身份披露。 目前还没有直接适用于纵向合成数据集的隐私模型。上述公开的统一模型将扩展到每个患者具有多个异质事件的纵向数据。在披露控制文献中使用的现有方法将被纳入合成数据隐私模型。 效用度量可以用于多种目的，例如模型优化和合成数据集评估，以接受或拒绝特定的生成数据集。 在该计划的这一部分中，将根据经验评估当前的效用指标。结果将澄清哪些实用性指标对优化有用，以及合成数据集的接受/拒绝。 目前，在评价综合纵向数据的效用方面缺乏工作。简单的方法，如k阶马尔可夫链之间的一致性捕获一些结构属性，但不提供相关的分析工作量的措施。该研究计划将扩展和评估纵向数据的实用性指标。风险效用优化通过适当定义的隐私和效用度量，可以定义组合的风险效用度量，并将其用作SDG算法的优化标准。这将确保生成的合成数据通过构造满足两个标准。将根据用于健康数据的常见SDG算法对此类措施进行评估。

Access to health data for secondary purposes remains a challenge because of privacy concerns. Synthetic data generation (SDG) has been proposed to enable data sharing that is believed to have low identification risks because there is no one-to-one mapping to real individuals. However, if the generative models used to generate synthetic data are overfit, or if a dataset is categorical with a small number of possible combinations of values, then real records may be generated. The adoption of SDG will also depend on demonstrating the utility of the generated data. Utility is broadly defined as the ability to replicate the conclusions from the analysis of real data on synthetic data. SDG needs to simultaneously optimize on privacy and utility. However, thus far SDG loss functions have largely been focused on maximizing utility, and privacy risks are often assessed after the data are generated. The purpose of this program is to develop a unified privacy framework for SDG, and to evaluate and improve current utility metrics. These results would then be used to define and test a combined loss metric that can be applied to optimize the generation of synthetic data which allows for the simultaneous management of privacy and utility. Privacy Evaluation Our focus in this program will be on identity disclosure conditional on attribute disclosure and membership disclosure. We will develop and validate a unified risk model that integrates identity, attribute, and membership disclosure. Currently there are no privacy models that are directly applicable to longitudinal synthetic datasets. The unified model of disclosure above will be extended to longitudinal data with multiple heterogeneous events per patient. Existing approaches used in the disclosure control literature will be incorporated into the synthetic data privacy model. Utility Evaluation Utility metrics can serve multiple purposes such as model optimization and synthetic dataset evaluation to accept or reject specific generated datasets. In this part of the program, current utility metrics will be empirically evaluated. The results will clarify which utility metrics are useful for optimization, and synthesized dataset acceptance/rejection. Currently, there has been a dearth of work on evaluating the utility of synthetic longitudinal data. Simple approaches such as concordance between k-order Markov chains capture some structural properties, but do not provide measures related to analytic workloads. This program of research will extend and evaluate the utility metrics for longitudinal data. Risk-Utility Optimization With appropriately defined privacy and utility metrics, a combined risk-utility measure can be defined and used as an optimization criterion for SDG algorithms. This will ensure that generated synthetic data satisfy both criteria by construction. Such a measure will be evaluated on common SDG algorithms used on health data.