Hybrid Quantum Transformer Architectures in Genomics

变换器和大型语言模型在处理基于语言的信息方面显示出巨大的潜力。它们在连接大型复杂数据集方面的优势使其上级传统的神经网络。最近有研究表明，transformers（Google的BERT模型）也可以应用于基因组数据集（DNA BERT）。大部分DNA（约98%）是非编码的（ncDNA），仅提供如何读取2%的编码DNA的指令，该编码DNA首先转录成mRNA，然后翻译成蛋白质。然而，许多复杂的医疗状况和心理健康状况都与ncDNA内的突变有关。非编码DNA（ncDNA）在统计上与人类语言非常相似。这已经被DNA BERT利用了。已有研究表明，传统BERT模型的最后一层Transformer可以用一个量子电路来代替，从而完成分类任务。本项目的目的是探索各种量子Transformer混合模型，并将其与经典模型进行比较。我们还将研究将基因组数据编码到量子态矢量中的更有效的方法，并可能考虑几何原理，不同的状态矢量编码和压缩方法。其目标是预测ncDNA中与医疗条件相关的功能区域。基因组学是一个快速增长的细分市场，在未来的药物发现中至关重要。虽然ncDNA是新疗法的更具挑战性的靶标，但研究表明存在可以结合这些区域并阻断这些区域的选择性分子。我们的目的是创建一个量子计算平台，支持ncDNA序列内疾病相关功能区域的识别。

Transformers and large language models have shown great potential in processing language-based information. Their strenth in connecting large and complex date sets makes them superior to conventional neural networks. Recently it has been shown that transformers (Google's BERT model) can also be applied to genomics datasets (DNA BERT). The majority of DNA (about 98%) is non-coding (ncDNA) and merely provides instructions on how to read the 2% of coding DNA that is first transcribed into mRNA and then translated into proteins. However, many complex medical conditions and also mental health conditions are associated with mutations within the ncDNA.The non-coding DNA (ncDNA) is statistically very similiar to human languages. This has been exploited with the DNA BERT. It has already been shown that the final transformer layer of the convential BERT model can be replaced with a variational quantum circuit to undertake classification tasks.The aim of our project is to explore various quantum transformer hybrids models and benchmark them against their classical counterparts. We will also investigate more efficient ways of encoding genomic data into the quantum state vector and may consider geometric principles, different state vector encoding and compression methods in this endeavour. The goal is to make predictions on functional regions within the ncDNA that have relevance for medical conditions.Genomics is a rapidly growing market segment and is essential in future drug discovery. Whilst ncDNA is a more challenging target for novel therapeutics, research has shown that selective molecules exist that can bind to these regions and block these regions. Our intention is to create a quantum computational platform that supports the identification of disease-relevant functional regions within ncDNA sequences.