Tuning Large language models to read biological literature

在这一应用中，我们关注两个相关的生物信息学挑战，这两个挑战需要从大规模的生物和生物医学文献中进行解释和知识提取。首先，基因/基因组数据库存储关于基因功能的信息，这些信息最终来自科学实验，并在出版物中报告结果。对于人类馆长来说，阅读所有相关的科学文献，解释关于基因产品的功能或定位的报道，并分配特定的受控词汇术语(例如，基因本体论术语)或简短的自由文本描述(基因名称或产品描述)，这是非常耗时和昂贵的。我们的团队和其他人为新的目的开发和应用管道来重新分析MS数据，包括注释基因组、发现翻译后修饰和建立物种或组织的定量图谱等。解释最初的实验设计、样本说明和软件参数是一个主要瓶颈，目前这些内容是在提交给档案库的自由文本块中或在所附文章的材料和方法部分中描述的。对于这两个挑战，我们认为，随着最近大型语言模型(LLM)的非凡改进，可以对它们进行再培训和利用，以消除从文献中提取知识的瓶颈。我们的团队在生物信息学和机器学习方面拥有重要的专业知识，但到目前为止在自然语言处理(NLP)方面的专业知识有限。在这项国际合作应用中，我们正在与宾夕法尼亚大学(UPenn)人工智能和NLP领域的一个领先团队合作。UPenn团队将使用我们团队多年积累的培训数据，帮助我们指导重新培训开源LLM的最佳方法。我们将为这两个挑战领域编写开放源代码，并制定更长期的计划，在我们发挥主导作用的主要国际数据库和财团的背景下将这些代码投入生产。

In this application, we focus on two related bioinformatics challenges that require interpretation and knowledge extraction from biological and biomedical literature at great scale.First, gene/genome databases store information on gene function, which is ultimately derived from scientific experiments with results reported in publications. It is exceptionally time-consuming and expensive for human curators to read all relevant scientific literature, interpret what has reported about the function or localisation of gene products, and assign specific controlled vocabulary terms (e.g. Gene Ontology terms) or short free text descriptions (gene names or product descriptions.Second, there are enormous volumes of raw data sets accompanying scientific publications, which are deposited in archival databases from expensive omics experiments, including mass spectrometry (MS) proteomics. Our group and others develop and apply pipelines for re-analysing MS data for new purposes, including annotating genomes, discovery of post-translational modifications and building quantitative atlases of species or tissues amongst others. There is a major bottleneck interpreting the original experimental design, sample descriptions and software parameters, which are currently described in blocks of free text submitted to the archival repository or within Materials and Methods sections of accompanying articles. For both challenges, we believe that with the recent extraordinary improvements in large language models (LLMs), they can be retrained and harnessed for these tasks, to remove the bottleneck in knowledge extraction from literature. Our group has significant expertise in bioinformatics and machine learning, but limited expertise in natural language processing (NLP) to date. In this international partnering application, we are collaborating with a leading group in artificial intelligence and NLP from the University of Pennsylvania (UPenn). The UPenn team will help to guide us in the optimal approach for re-training open source LLMs, using training data that our team has amassed over many years. We will produce open source code for the two challenge areas, with a longer term plan to put these into production within the context of major international databases and consortia, within which we have leading roles.