Artificial Intelligence for Missing Data Imputation in Electronic Medical Records

多年来，英国和加拿大的卫生系统广泛使用电子病历(EMR)，将其作为其业务的组成部分。然而，虽然存在数字记录的数据，但将其用作“学习卫生系统”的基础，通过整理和检查数据和证据来改进所有这些领域，从而不断改进患者体验、医院运作和护理质量。然而，现实世界中的电子病历数据可能非常难以处理。造成这些困难的一个重要因素是数据质量。数据丢失是一个特别的问题，某些记录的漏失率在10%-30%之间。正确解决电子病历数据中的丢失数据问题很复杂，因为很难区分真正的丢失数据(数据没有记录到系统中)和不适用的响应(例如，测试不合适，因此没有进行测试)。数据可以是随机丢失(MAR)或非随机丢失(MNAR)，在后者中，有一个决定丢失模式的潜在因素。因此，某些类型的失眠可能是“信息性的”，因为如果临床医生决定不安排某些检查，它表明了对患者所感知的健康状态的某种隐含的信念。如果不能解释这些偏见的来源，可能会导致错误的推断。人工智能技术被视为解锁电子医疗记录中信息财富的重要工具。该项目将有助于这些技术的成熟，以解决电子病历数据集在现实世界中的复杂性。这里提出的研究将开发用于数据补偿的算法，寻求更健壮、可靠和可推广的算法。我们最初选择将重点放在脓毒症的自动诊断上，这是生物医学研究的一个紧迫领域，因为仅在英国每年就有大约44,000人死于脓毒症。因此，通过将基于机器学习的现代方法应用于大型电子病历数据集，我们承诺以一种独特的方式解决这个问题，这可能会对现实世界产生有意义的影响。然而，由于许多人工智能预测模型需要完整的数据集作为输入，处理缺失数据的一种流行策略涉及“数据填补”，即使用算法来填充缺失的数据值。这些方法的复杂性各不相同，从简单地用整个数据集上的平均观测值填充缺失值，到尝试得出数据中的基本模式的更高级的方法。然而，目前的许多推算方法只针对某些类型的EMR数据(例如，分子测量的临床时间序列)，不能解释偏差的来源，也不能提供关于推算数据质量的确定性度量。这个项目的总体目标是开发新的机器学习方法，用于EMR中考虑到偏差和统计不确定性的缺失数据补充。

Health systems in the UK and Canada have made extensive use of Electronic Medical Records (EMR) for many years as an integral part of their operations. However, whilst digitally recorded data exists, their use as the basis of a "learning health system" whereby continuous improvements in patient experience, hospital operations, and quality of care has are made by collating and examining data and evidence to improve all these areas. However, real-world EMR data can be very challenging to handle.One significant contribution to these difficulties is data quality. Missing data is a particular issue, with rates of missingness of between 10-30% for some records. Properly addressing the missing data issue in EMR data is complicated by the fact that it can be difficult to differentiate between genuine missing data (data was not recorded into the system) and a non-applicable response (e.g. the test was not appropriate therefore it was not done). Data can be missing-at-random (MAR) or missing-not-at-random (MNAR) where, in the latter, there is an underlying factor that determines the missingness patterns. Certain types of missingness can therefore be "informative" since, if a clinician decided not to order certain tests, it indicates a certain implicit belief about the perceived health state of the patient. Failure to account for these sources of bias may lead to incorrect inferences.Artificial Intelligence technologies are seen as an important tool in unlocking the information wealth held in our electronic medical records. This project will contribute to the maturation of these technologies to account for the real-world complexities of EMR datasets. The research proposed here will develop algorithms for data imputation that seek to be more robust, reliable and generalisable. We have chosen to initially focus on automated sepsis diagnosis, a pressing area of biomedical research given that sepsis accounts for around 44,000 deaths each year in the UK alone. Therefore, by applying modern approaches based on machine learning to large EMR datasets we promise to tackle this problem in a unique way that could have meaningful real-world impact.However, as many AI prediction models require complete datasets as input, one popular strategy for handling missing data involves "data imputation", whereby an algorithm is used to fill in missing data values. These methods vary in complexity from simply filling in missing values with the average observed values over the entire dataset through to more advanced methods that attempt to elicit the underlying patterns in the data. However, many current imputation methods are designed for only certain types of EMR data (e.g. clinical time series of molecular measurements) and fail to account for sources of bias and provide measures of certainty about the quality of the imputed data. The overall goal of this project is to develop novel machine learning methods for missing data imputation in EMRs that account for biases and statistical uncertainty in the imputation.