On the application of BERT models for nanopore methylation detection
On the application of BERT models for nanopore methylation detection
复制标题
BERT模型在纳米孔甲基化检测中的应用
DOI：
10.1109/bibm52615.2021.9669841
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Imoto Seiya
中科院分区：
文献类型：
--
作者：
Zhang Yao-Zhong;Yamaguchi Kiyoshi;Hatakeyama Sera;Furukawa Yoichi;Miyano Satoru;Yamaguchi Rui;Imoto Seiya
DNA甲基化是一种常见的核苷酸修饰，与基因表达和衰老等多种生物学过程相关。纳米孔测序通过搜索特定的电流信号偏移提供了直接的检测方法。最近，基于模型的方法，特别是使用深度学习模型的方法，在纳米孔甲基化检测方面取得了显着的性能改进。在这项工作中，我们探索使用变压器双向编码器表示（BERT）的非递归神经网络结构来完成任务，这为最先进的双向递归神经网络（biRNN）提供了一种替代的快速推理模型。此外，我们提出了一个改进的BERT模型，相对位置表示和中心隐藏单元连接，考虑到特定的任务特性建模。我们在不同基序和甲基转移酶的R9基准数据集上评估了所提出的模型。实验结果表明，改进后的BERT模型可以获得与现有的biRNN模型相当甚至更好的结果，同时模型推理速度更快。
DNA methylation is a common nucleotide modification, which is associated with various biological processes, such as gene expression and aging. Nanopore sequencing provides a direct detecting approach through searching specific current signal shifts. Recently, model-based approaches, especially those using deep learning models, have achieved significant performance improvements on nanopore methylation detection. In this work, we explore using the non-recurrent neural network structure of Bidirectional Encoder Representations from Transformers (BERT) for the task, which provides an alternative fast inference model to the state-of-the-art bi-directional Recurrent Neural Network (biRNN). In addition, we propose a refined BERT model with relative position representation and center hidden units concatenation, which takes account of the task-specific characters into modeling. We evaluate the proposed models on the R9 benchmark datasets of different motifs and methyltransferases. The experiment results show that the refined BERT model can achieve competitive or even better results than the state-of-the-art biRNN model, while the model inference speed is faster.