BiomedGPT: A Unified and Generalist Biomedical Generative Pre-trained Transformer for Vision, Language, and Multimodal Tasks
BiomedGPT: A Unified and Generalist Biomedical Generative Pre-trained Transformer for Vision, Language, and Multimodal Tasks
复制标题
DOI：
10.48550/arxiv.2305.17100
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Kai Zhang-;Jun Yu;Zhilin Yan;Yixin Liu-;Eashan Adhikarla;S. Fu;Xun Chen;Chen Chen-Chen;Yuyin Zhou;Xiang Li;Lifang He;B. Davison;Quanzheng Li;Yong Chen;Hongfang Liu;Lichao Sun
中科院分区：
文献类型：
--
作者：
Kai Zhang-;Jun Yu;Zhilin Yan;Yixin Liu-;Eashan Adhikarla;S. Fu;Xun Chen;Chen Chen-Chen;Yuyin Zhou;Xiang Li;Lifang He;B. Davison;Quanzheng Li;Yong Chen;Hongfang Liu;Lichao Sun
在本文中，我们介绍了一个统一的和通用的BiomedGenerative P re-trained T transformer（BiomedGPT）模型，该模型利用大型和多样化数据集的自我监督来接受多模态输入并执行一系列下游任务。我们的实验表明，BiomedGPT提供了生物医学数据的广泛和包容性的表示，在五个不同的任务中，超过了大多数先前的最先进的模型，其中20个公共数据集跨越了15个独特的生物医学模式。通过消融研究，我们还展示了我们的多模式和多任务预训练方法在将知识转移到以前看不见的数据方面的有效性。总的来说，我们的工作在开发生物医学的统一和通用模型方面迈出了重要的一步，对改善医疗保健结果具有深远的影响。
In this paper, we introduce a unified and generalist Biomed ical G enerative P re-trained T ransformer ( BiomedGPT ) model, which leverages self-supervision on large and diverse datasets to accept multi-modal inputs and perform a range of downstream tasks. Our experiments demonstrate that BiomedGPT delivers expansive and inclusive representations of biomedical data, outperforming the majority of preceding state-of-the-art models across five distinct tasks with 20 public datasets spanning over 15 unique biomedical modalities. Through the ablation study, we also showcase the efficacy of our multi-modal and multi-task pretraining approach in transferring knowledge to previously unseen data. Overall, our work presents a significant step forward in developing unified and generalist models for biomedicine, with far-reaching implications for improving healthcare outcomes.