Transfer Learning for Digital Curation of the EMR Clinical Narrative

用于 EMR 临床叙述数字化管理的迁移学习

• 批准号: 10468604
• 负责人: GUERGANA K. SAVOVA
• 金额: $ 37.61万
• 依托单位: BOSTON CHILDREN'S HOSPITAL
• 依托单位国家: 美国
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-08-12 至 2025-05-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10468604
• 关键词: Adverse event; Apache; Appearance; Area; Artificial Intelligence; Automated Annotation; Biomedical Research; Brain Aneurysms; Classification; Clinical; Clinical Investigator; Communities; Computer Vision Systems; Computerized Medical Record; Coupled; Data; Data Science; Data Set; Databases; Development; Disease; E-learning; Engineering; Epigenetic Process; Evaluation; Event; Genetic; Health; Hepatotoxicity; Image; Information Retrieval; Intuition; Investigation; Label; Language; Machine Learning; Medicine; Methodology; Methods; Methotrexate; Modeling; Multiple Sclerosis; Natural Language Processing; Neural Network Simulation; Patients; Performance; Pharmaceutical Preparations; Phenotype; Procedures; Public Health; Publications; Publishing; Rare Diseases; Reporting; Research; Resources; Rheumatoid Arthritis; Semantics; Severities; Signs and Symptoms; Source; Speech; Structure; Supervision; System; Techniques; Testing; Text; Training; Translational Research; Vision; Work; autism spectrum disorder; base; comorbidity; computing resources; deep learning; deep neural network; digital; imaging Segmentation; improved; informatics tool; learning strategy; machine learning method; multitask; neural network; next generation; novel; open source; phenotypic data; relating to nervous system; response; support vector machine; transfer learning; unstructured data

Project Summary This proposal is in response to PAR 18-796 to seek support for advancing methodologies for a transfer learning framework for the digital curation of the Electronic Medical Records (EMR) clinical narrative. In the current era of increasing importance of Artificial Intelligence (AI) in biomedicine, our proposal tackles a critical AI component – automated annotation of health-related text. Since 2015 the development and application of machine learning (ML) methods has exploded propelled by the convergence of plentiful digitized unstructured data (text, speech, images), hardware and the refinement of neural networks or deep learning. 2018 marked a turning point in Natural Language Processing (NLP), particularly transfer learning through pre-trained models like Universal Language Model Fine-tuning for Text Classification, Allen AI's ELMO, OpenAI's Open-GPT. In November 2018, Google published the Bidirectional Encodings Representations from Transformers (BERT), a transformer-based model pre-trained on massive general text databases (3.3B words total). The publication reported using BERT representations to build classifiers for 11 NLP tasks which outperformed the state-of-the- art (SOTA) with large margins. The NLP research community jumped to the idea of exploring this new framework but quickly came to the realization that building BERT-style models from scratch is affordable and feasible to only a few. Thus, research investigation proceeded in the direction of using these gigantic models as resources for language representations. Scientific efforts focused on pre-trained models (e.g. BERT) as a source of extracting high quality language features or fine-tuning on a specific task, i.e. using a model as a checkpoint and re-training with much smaller amounts of task-specific data to produce predictions by typically adding one fully-connected layer on top of the representations and training for a few epochs. This general watershed shift in NLP to transfer learning which parallels the developments in computer vision a few years ago coupled with our latest work brings to the forefront a critical NLP research topic ripe for exploration – a transfer learning framework for the digital curation of the EMR clinical narrative. The proposed work is research of novel scientific methods for extracting detailed information from health-related text especially the EMR, the major source of phenotype data for patients. Precise phenotype information is needed to advance translational research, particularly to unravel the effects of genetic, epigenetic, and systems changes on responsiveness. This research is in line with the latest developments in neural deep learning approaches and AI in general and is expected to enhance biomedical research and through that the health of the public.

项目概要 该提案是对 PAR 18-796 的回应，旨在寻求对推进转移方法的支持 电子病历 (EMR) 临床叙述的数字管理学习框架。在 当前人工智能 (AI) 在生物医学中的重要性日益增加，我们的提案解决了一个关键问题 AI 组件 – 健康相关文本的自动注释。 2015年以来的开发应用 大量数字化非结构化数据的融合推动了机器学习 (ML) 方法的爆炸式增长 数据（文本、语音、图像）、硬件以及神经网络或深度学习的细化。 2018年标志着 自然语言处理 (NLP) 的转折点，特别是通过预训练模型进行迁移学习 比如用于文本分类的通用语言模型微调、Allen AI 的 ELMO、OpenAI 的 Open-GPT。在 2018 年 11 月，Google 发布了 Transformers 的双向编码表示法 (BERT)， 基于 Transformer 的模型在海量通用文本数据库（总共 3.3B 字）上进行了预训练。刊物 报告使用 BERT 表示为 11 个 NLP 任务构建分类器，其性能优于当前水平 艺术（SOTA）利润丰厚。 NLP 研究界立即萌生了探索这一新领域的想法 但很快就意识到从头开始构建 BERT 风格的模型是经济实惠的，而且 只有少数人可行。因此，研究调查朝着使用这些巨大模型的方向进行。 作为语言表示的资源。科学工作重点关注预训练模型（例如 BERT）作为 提取高质量语言特征或对特定任务进行微调的来源，即使用模型作为 检查点并使用更少量的特定于任务的数据进行重新训练，以产生通常的预测 在表示之上添加一个全连接层并进行几个时期的训练。这个一般 NLP 向迁移学习的分水岭转变，与计算机视觉几年来的发展并行 之前结合我们的最新工作，我们提出了一个值得探索的关键 NLP 研究主题—— 用于 EMR 临床叙述数字化管理的迁移学习框架。拟议的工作是研究 从健康相关文本（尤其是 EMR）中提取详细信息的新颖科学方法 患者表型数据的主要来源。需要精确的表型信息来推进转化 研究，特别是揭示遗传、表观遗传和系统变化对反应能力的影响。 这项研究符合神经深度学习方法和人工智能的最新发展 预计将加强生物医学研究，并通过其提高公众的健康。