MSA-Conditioned Generative Protein Language Models for Fitness Landscape Modelling and Design
MSA-Conditioned Generative Protein Language Models for Fitness Landscape Modelling and Design
复制标题
用于健身景观建模和设计的 MSA 条件生成蛋白语言模型
DOI：
--
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
David T. Jones
中科院分区：
文献类型：
--
作者：
Alex Hawkins;David T. Jones
最近，一些研究已经成功地应用了完全数据驱动的蛋白质设计方法，该方法基于进化相关序列家族分布的学习生成模型。语言建模技术有望在蛋白质空间中推广这种设计范式，然而，在很大程度上忽略了多个序列比对中的丰富进化信号，并依赖于微调来适应特定家族的学习分布。受最近基于对齐的语言模型发展的启发，以MSA Transformer为例，我们提出了一种新的基于对齐的生成模型，该模型结合了输入MSA编码器和自回归序列解码器，产生了一个可以明确地以进化背景为条件的生成序列模型。为了测试这种基于生成式msa的方法在与设计相关的环境中的好处，我们专注于无监督适应度景观建模的问题。在三个异常不同的适应度景观中，我们发现证据表明，与使用非生成掩码语言模型计算的分数相比，直接对整个序列空间的分布进行建模可以改进变体适应度的无监督预测。我们认为，将进化信息的显式编码与生成解码器在序列空间上的分布表示相结合，提供了一个强大的框架，可以推广传统的基于家族的生成模型。
Recently a number of works have demonstrated successful applications of a fully data-driven approach to protein design, based on learning generative models of the distribution of a family of evolutionarily related sequences. Language modelling techniques promise to generalise this design paradigm across protein space, however have for the most part neglected the rich evolutionary signal in multiple sequence alignments and relied on fine-tuning to adapt the learned distribution to a particular family. Inspired by the recent development of alignment-based language models, exemplified by the MSA Transformer, we propose a novel alignment-based generative model which combines an input MSA encoder with an autoregressive sequence decoder, yielding a generative sequence model which can be explicitly conditioned on evolutionary context. To test the benefits of this generative MSA-based approach in design-relevant settings we focus on the problem of unsupervised fitness landscape modelling. Across three unusually diverse fitness landscapes, we find evidence that directly modelling the distribution over full sequence space leads to improved unsupervised prediction of variant fitness compared to scores computed with non-generative masked language models. We believe that combining explicit encoding of evolutionary information with a generative decoder’s representation of a distribution over sequence space provides a powerful framework generalising traditional family-based generative models.