Hybrid Quantum Transformer Architectures in Genomics

点击翻译按钮获取中文摘要

Transformers and large language models have shown great potential in processing language-based information. Their strenth in connecting large and complex date sets makes them superior to conventional neural networks. Recently it has been shown that transformers (Google's BERT model) can also be applied to genomics datasets (DNA BERT). The majority of DNA (about 98%) is non-coding (ncDNA) and merely provides instructions on how to read the 2% of coding DNA that is first transcribed into mRNA and then translated into proteins. However, many complex medical conditions and also mental health conditions are associated with mutations within the ncDNA.The non-coding DNA (ncDNA) is statistically very similiar to human languages. This has been exploited with the DNA BERT. It has already been shown that the final transformer layer of the convential BERT model can be replaced with a variational quantum circuit to undertake classification tasks.The aim of our project is to explore various quantum transformer hybrids models and benchmark them against their classical counterparts. We will also investigate more efficient ways of encoding genomic data into the quantum state vector and may consider geometric principles, different state vector encoding and compression methods in this endeavour. The goal is to make predictions on functional regions within the ncDNA that have relevance for medical conditions.Genomics is a rapidly growing market segment and is essential in future drug discovery. Whilst ncDNA is a more challenging target for novel therapeutics, research has shown that selective molecules exist that can bind to these regions and block these regions. Our intention is to create a quantum computational platform that supports the identification of disease-relevant functional regions within ncDNA sequences.