Learning Universal Patient Representations with Hierarchical Transformers

使用分层转换器学习通用患者表示

• 批准号: 10587270
• 负责人: Timothy A Miller
• 金额: $ 64.82万
• 依托单位: BOSTON CHILDREN'S HOSPITAL
• 依托单位国家: 美国
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-02-07 至 2027-07-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10587270
• 关键词: Accident and Emergency department; Address; Algorithms; Architecture; Area; Chest; Classification; Clinical; Clinical Research; Code; Competence; Complex; Computers; Data; Data Aggregation; Data Set; Databases; Decision Making; Development; Diagnosis; E-learning; Electronic Health Record; Experimental Designs; Funding; Goals; Health Status; Image; Individual; Injury; Intensive Care Units; Knowledge; Learning; Length; Link; Machine Learning; Maps; Medical; Methods; Modality; Modeling; Output; Patient Care; Patient Education; Patients; Performance; Phenotype; Privatization; Process; Productivity; Research; Severities; Source; Structure; Supervision; System; Techniques; Terminology; Text; Time; Training; Unified Medical Language System; Work; data fusion; experimental study; frontier; health data; improved; innovation; language training; light weight; machine learning algorithm; multimodality; multiple data types; neural; neural model; novel; patient prognosis; point of care; prevent; prognostic; structured data; usability

Project Summary Electronic health records contain a wealth of information about patient health status that can be mined for multiple purposes, including clinical research and improved decision-making at the point of care. This information can be represented as structured variables, unstructured text, and images, among other data types. In this work, we develop new models for representing the unstructured text that take advantage of powerful neural models called pre-trained transformers. We propose to make these models usable for much longer texts by adding hierarchical layers to operate over summaries of smaller chunks of text, and shrinking the size of the encoder that operates on smaller chunks. First, we develop a smaller encoder for sentence and paragraph-sized texts, by using a technique called extreme distillation that trains smaller models from the output of larger models. We also propose to pre-train hierarchical models for text, by taking advantage of smaller encoders like that from the first aim. We take advantage of both public and private datasets and experiment with different pre-training tasks and architectures. Our final aim proposes to combine representations learned from text with those from the more mature areas of structured data and images. We design experiments that answer the question of how best to merge these different information sources, and apply them to important clinical classification use cases that are likely to require multiple information sources for accurate performance. Specifically, we address the clinical tasks of predicting injury severity in emergency departments, and predicting diagnosis and prognosis of patients in intensive care units.

项目摘要 电子健康记录包含了大量关于患者健康状况的信息，这些信息可以被挖掘出来， 多种用途，包括临床研究和改善护理决策。这 信息可以表示为结构化变量、非结构化文本和图像以及其他数据 类型在这项工作中，我们开发了新的模型来表示非结构化文本， 强大的神经模型，称为预先训练的变压器。我们建议使这些模型可用于 通过添加层次结构层来操作较小文本块的摘要， 对较小块进行操作的编码器的大小。首先，我们开发了一个更小的句子编码器， 段落大小的文本，通过使用一种称为极端蒸馏的技术， 大型模型的输出。我们还建议通过利用 像第一个目标那样的小编码器。我们利用公共和私人数据集， 尝试不同的预训练任务和架构。我们的最终目标是将联合收割机 从文本中学习的表示与来自结构化数据和图像的更成熟领域的表示进行比较。我们 设计实验，回答如何最好地合并这些不同的信息源的问题， 将它们应用于可能需要多个信息源的重要临床分类用例 以实现准确的性能。具体来说，我们解决的临床任务，预测损伤严重程度在紧急情况下， 部门，并预测重症监护病房患者的诊断和预后。