Stochastic Deep Learning for Electronic Health Records: Localizing Learning with Massive and Fragmented Data

随着新世纪的到来，计算机技术与医学实践的结合使 科学家们将收集大量的电子健康记录(EHR)，同时，深入 学习已经发展成为海量数据分析的主要工具。然而，EHR的数据是 异质性[对不同的患者群体有很大不同]和支离破碎的[由高 缺失值比例]，这对《公约》的适用性和普遍性构成了重大障碍 当前的深度神经网络。该项目旨在构建一种基于新型健康预测系统 具有海量、异质和碎片化数据的随机神经网络(StoNet)，而 在训练系统时考虑整合组学、影像和EHR数据。StoNet是 由许多简单回归组成的公式；它渐近等同于深层神经 当训练样本量变大时，网络(DNN)在函数逼近中，但其结构是 更加灵活地处理复杂的电子病历数据。该网络由一种自适应随机网络来训练 梯度马尔可夫链蒙特卡罗(MCMC)算法。通过利用灵活的 StoNet和成熟的自适应随机梯度MCMC算法，本项目提供了一种 针对海量、异质和零散的电子病历数据的深度学习的严格统计框架。 我们证明了StoNet形成了从线性模型到深度学习的桥梁，使许多理论成为可能 以及为将线性模型转移到深度学习而开发的方法。特别是，我们向您展示了 为带有套索惩罚的线性模型开发的稀疏学习理论可以转移到 StonNet，导致了一种创新的一致稀疏深度学习方法；我们解决了数据 用混合体替换StoNet的第一个隐层的每个回归的异质性问题 回归；我们通过用自适应随机训练StoNet来解决丢失数据的问题 梯度MCMC算法，其中缺失数据被归结为具有多个 归责方法。StoNet的马尔可夫结构使网络参数能够在本地 利用碎片化数据学习，并导致了一种创新的非线性充分降维方法 高维数据，促进不同类型数据在StoNet培训中的整合。我们还展示了 通过递归应用Eve定律，可以很容易地量化StoNet的预测不确定性。

Coming with the new century, integration of computer technology into medical practice has enabled scientists to collect massive volumes of electronic health records (EHR) and, in the meantime, deep learning has been developed as the major tool of massive data analysis. However, the EHR data are heterogeneous [varied much for different groups of patients] and fragmented [consisting of a high proportion of missing values], which poses a significant barrier to the applicability and generalizability of current deep neural networks. This project aims to build a health prediction system based on a new type of stochastic neural network (StoNet) with massive, heterogeneous, and fragmented data, while considering integration of the omics, imaging and EHR data in training the system. The StoNet is formulated as a composition of many simple regressions; it is asymptotically equivalent to the deep neural network (DNN) in function approximation as the training sample size becomes large, but its structure is more flexible for dealing with the complexity of EHR data. The StoNet is trained by an adaptive stochastic gradient Markov chain Monte Carlo (MCMC) algorithm. By leveraging on the flexible structure of the StoNet and the sophisticated adaptive stochastic gradient MCMC algorithm, this project provides a rigorous statistical framework for deep learning with massive, heterogeneous and fragmented EHR data. We show that the StoNet forms a bridge from linear models to deep learning, enabling many of the theory and methods developed for linear models to be transferred to deep learning. In particular, we show the sparse learning theory developed for linear models with the Lasso penalty can be transferred to the StonNet, leading to an innovative consistent sparse deep learning method; we address the data heterogeneity issue by replacing each regression of the first hidden layer of the StoNet by a mixture regression; and we address the missing data issue by training the StoNet with an adaptive stochastic gradient MCMC algorithm where the missing data are imputed as for a linear model with multiple imputation methods. The Markovian structure of the StoNet enables the network parameters to be locally learned with fragmented data and leads to an innovative way for nonlinear sufficient dimension reduction of high-dimensional data, facilitating integration of different types of data in StoNet training. We also show the prediction uncertainty of the StoNet can be easily quantified with a recursive application of Eve's law.