Bidirectional Representation Learning From Transformers Using Multimodal Electronic Health Record Data to Predict Depression.
Bidirectional Representation Learning From Transformers Using Multimodal Electronic Health Record Data to Predict Depression.
复制标题
DOI：
10.1109/jbhi.2021.3063721
复制
发表时间：
2021-08
影响因子：
7.7
通讯作者：
Arnold CW
中科院分区：
文献类型：
--
作者：
Meng Y;Speier W;Ong MK;Arnold CW
机器学习算法的进步对使用电子健康记录（EHR）数据构建的表示学习，分类和预测模型产生了有益的影响。已作出努力，提高模型的总体性能，并改善其可解释性，特别是在决策过程中。在这项研究中，我们提出了一个时间深度学习模型，对具有Transformer架构的EHR序列进行双向表示学习，以预测未来的抑郁症诊断。该模型能够聚合来自EHR的五个异构且高维的数据源，并以时间方式处理它们，以便在各个预测窗口进行慢性疾病预测。我们应用了当前对EHR数据进行预训练和微调的趋势，以在慢性疾病预测方面优于当前最先进的技术，并展示了序列中EHR代码之间的潜在关系。与最佳基线模型相比，该模型在抑郁症预测中产生了最高的精确-回忆曲线下面积（PRAUC），从0.70增加到0.76。此外，每个序列中的自我注意权重定量地反映了各个编码之间的内在联系，提高了模型的可解释性。这些结果表明，该模型能够利用异构EHR数据来预测抑郁症，同时实现高准确性和可解释性，这可能有助于构建临床决策支持系统，在未来的慢性疾病筛查和早期检测。
Advancements in machine learning algorithms have had a beneficial impact on representation learning, classification, and prediction models built using electronic health record (EHR) data. Effort has been put both on increasing models’ overall performance as well as improving their interpretability, particularly regarding the decision-making process. In this study, we present a temporal deep learning model to perform bidirectional representation learning on EHR sequences with a transformer architecture to predict future diagnosis of depression. This model is able to aggregate five heterogenous and high-dimensional data sources from the EHR and process them in a temporal manner for chronic disease prediction at various prediction windows. We applied the current trend of pretraining and fine-tuning on EHR data to outperform the current state-of-the-art in chronic disease prediction, and to demonstrate the underlying relation between EHR codes in the sequence. The model generated the highest increases of precision-recall area under the curve (PRAUC) from 0.70 to 0.76 in depression prediction compared to the best baseline model. Furthermore, the self-attention weights in each sequence quantitatively demonstrated the inner relationship between various codes, which improved the model’s interpretability. These results demonstrate the model’s ability to utilize heterogeneous EHR data to predict depression while achieving high accuracy and interpretability, which may facilitate constructing clinical decision support systems in the future for chronic disease screening and early detection.