Navigating Chemical Space with Natural Language Processing and Deep Learning

利用自然语言处理和深度学习驾驭化学空间

• 批准号: EP/Y004167/1
• 负责人: Jiayun Pang
• 金额: $ 11.41万
• 依托单位: University of Greenwich
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2024
• 资助国家: 英国
• 起止时间: 2024 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FY004167%2F1
• 关键词: Navigating; Chemical; Space; Natural; Language

Natural language processing (NLP) lies at the intersection between linguistics and computer science which aims to process and analyse human language, typically provided as written text. NLP is now strongly focused on the use of machine learning for challenging tasks with some revolutionary algorithms having been developed in the last few years. They now underpin a wide range of real-life applications, such as ChatGPT, virtual assistants and automatic text completion when we write emails. Innovative research ideas often come from integrating techniques and concepts across disciplines. For this discipline-hopping grant, we would like to explore how Transformer models, a ground-breaking deep learning algorithm developed by Google in 2017 which fuels majority of the current cutting-edge research in NLP, can be adapted to solve research challenges in chemistry. Chemical structures are usually three dimensional. However, they are also often converted into sequences, called SMILES. SMILES has a simple vocabulary of chemical elements and bond symbols and a few grammatical rules of how the chemical elements are positioned. Owing to this direct analogy to text sequences, through SMILES it is possible to use NLP algorithms to analyse chemical structures in a similar fashion as they are used to analyse text. For the proposed research, Dr Pang, a chemist will work with Dr Vulic, an NLP and machine learning expert in order to get up to speed with the latest developments in the field of NLP and to examine their further applicability in her domain of expertise. We will explore and utilise a concept which is now pervasive in machine learning and NLP, termed transfer learning, which 1) pretrains large general-purpose models, and 2) fine-tunes (i.e., specialises) those general models for specific tasks and applications, where labelled data are expensive to create (as they require expert knowledge and complex annotation protocols) and thus inherently scarce. Specifically, we will pretrain Transformer models to learn a latent representation of the chemical space defined by tens of millions of SMILES. This learned latent representation can then be used to predict molecular properties for a given chemical structure during fine-tuning. The advantage of this type of approach is that the resulting machine learning models rely less on the so-called labelled data (molecules with experimentally determined properties), which are time-consuming or even impossible to generate in chemistry considering the associated cost and experimental challenges. We will aim to make the Transformer models more computationally efficient and accurate using two latest machine learning techniques, termed sentence encoding and contrastive learning. We hope that this new molecular representation can complement existing molecular representation methods and provide an alternative approach to evaluate molecular structures against their properties, which underpins many research and development tasks in the chemical and pharmaceutical industries.

自然语言处理（NLP）位于语言学和计算机科学之间的交叉点，旨在处理和分析人类语言，通常以书面文本的形式提供。NLP现在非常关注使用机器学习来完成具有挑战性的任务，在过去几年中开发了一些革命性的算法。它们现在支持广泛的现实生活应用程序，例如ChatGPT，虚拟助手和我们写电子邮件时的自动文本完成。创新的研究思路往往来自跨学科的技术和概念的整合。对于这个跨学科的资助，我们想探索Transformer模型，这是Google在2017年开发的一种突破性的深度学习算法，它为当前NLP的大部分前沿研究提供了动力，可以用来解决化学领域的研究挑战。化学结构通常是三维的。然而，它们也经常被转换成序列，称为SMILES。SMILES有一个简单的化学元素和键符号的词汇表，以及一些关于化学元素如何定位的语法规则。由于这种与文本序列的直接类比，通过SMILES，可以使用NLP算法以类似于分析文本的方式分析化学结构。对于拟议的研究，化学家Pang博士将与NLP和机器学习专家Vulic博士合作，以加快NLP领域的最新发展，并研究其在其专业领域的进一步适用性。我们将探索和利用一个现在在机器学习和NLP中普遍存在的概念，称为迁移学习，它1）预训练大型通用模型，2）微调（即，专门化）那些用于特定任务和应用的通用模型，其中标记数据创建起来是昂贵的（因为它们需要专业知识和复杂的注释协议），因此本质上是稀缺的。具体来说，我们将预训练Transformer模型，以学习由数千万个SMILES定义的化学空间的潜在表示。然后，这种学习的潜在表示可以用于在微调期间预测给定化学结构的分子性质。这种方法的优点是，所产生的机器学习模型较少依赖于所谓的标记数据（具有实验确定属性的分子），考虑到相关的成本和实验挑战，这些数据在化学中是耗时甚至不可能生成的。我们的目标是使用两种最新的机器学习技术，即句子编码和对比学习，使Transformer模型在计算上更加高效和准确。我们希望这种新的分子表示可以补充现有的分子表示方法，并提供一种替代方法来评估分子结构对其性质的影响，这是化学和制药工业中许多研究和开发任务的基础。