Recurrent Neural Networks and Their Applications to RNA Secondary Structure Inference
Recurrent Neural Networks and Their Applications to RNA Secondary Structure Inference
复制标题
循环神经网络及其在 RNA 二级结构推理中的应用
DOI：
10.13023/etd.2018.401
复制
发表时间：
2018
影响因子：
4.4
通讯作者：
Devin Willmott
中科院分区：
文献类型：
--
作者：
Devin Willmott
递归神经网络（RNN）是最先进的顺序机器学习工具，但由于通过RNN反向传播的梯度的指数增长或衰减，很难学习具有长程依赖性的序列。一些方法通过修改标准RNN架构来克服这个问题，以迫使递归权重矩阵W在整个训练过程中保持正交。本论文的前半部分提出了一种新的正交RNN架构，通过Cayley变换用反对称矩阵参数化来增强W的正交性。我们提出了通过Cayley变换进行反向传播的规则，展示了如何处理Cayley变换的奇异性，并将其在基准任务上的性能与其他正交RNN架构进行了比较。下半部分探讨了两种解决RNA二级结构推断问题的深度学习方法，并将其与标准结构推断工具最近邻热力学模型（NNTM）进行了比较。第一种使用RNN检测RNA结构中的配对或未配对核苷酸，然后将其转换为指导NNTM结构预测的合成辅助数据。第二种方法使用递归和卷积网络来直接推断RNA碱基对。在许多情况下，这些方法比NNTM结构预测提高了20-30个百分点。
OF DISSERTATION Recurrent Neural Networks and Their Applications to RNA Secondary Structure Inference Recurrent neural networks (RNNs) are state of the art sequential machine learning tools, but have difficulty learning sequences with long-range dependencies due to the exponential growth or decay of gradients backpropagated through the RNN. Some methods overcome this problem by modifying the standard RNN architecure to force the recurrent weight matrix W to remain orthogonal throughout training. The first half of this thesis presents a novel orthogonal RNN architecture that enforces orthogonality of W by parametrizing with a skew-symmetric matrix via the Cayley transform. We present rules for backpropagation through the Cayley transform, show how to deal with the Cayley transform’s singularity, and compare its performance on benchmark tasks to other orthogonal RNN architectures. The second half explores two deep learning approaches to problems in RNA secondary structure inference and compares them to a standard structure inference tool, the nearest neighbor thermodynamic model (NNTM). The first uses RNNs to detect paired or unpaired nucleotides in the RNA structure, which are then converted into synthetic auxiliary data that direct NNTM structure predictions. The second method uses recurrent and convolutional networks to directly infer RNA base pairs. In many cases, these approaches improve over NNTM structure predictions by 20-30 percentage points.
影响因子：
6.8
作者：
Gutell, RR;Lee, JC;Cannone, JJ
通讯作者：
Cannone, JJ