Visual Parsing with Query-Driven Global Graph Attention (QD-GGA): Preliminary Results for Handwritten Math Formula Recognition
Visual Parsing with Query-Driven Global Graph Attention (QD-GGA): Preliminary Results for Handwritten Math Formula Recognition
复制标题
DOI：
10.1109/cvprw50498.2020.00293
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Mahshad Mahdavi;Leilei Sun;R. Zanibbi
中科院分区：
文献类型：
--
作者：
Mahshad Mahdavi;Leilei Sun;R. Zanibbi
我们提出了一种基于手写数学公式的卷积神经网络的新视觉解析方法。查询驱动的全局图注意（QD-GGA）解析模型采用多任务学习，并使用单个功能表示来定位，分类和关联符号。首先，通过公式的手写笔触计算出视线（LOS）图。其次，使用特定于特定特征的特征过滤器（即注意）在单个进料前传球中获得了LOS节点和边缘的类别分布。最后，从加权图中提取了最大生成树（MST）。我们的初步结果表明，这是一种有希望的新方法，用于视觉解析手写配方。我们的数据和源代码公开可用。
We present a new visual parsing method based on convolutional neural networks for handwritten mathematical formulas. The Query-Driven Global Graph Attention (QD-GGA) parsing model employs multi-task learning, and uses a single feature representation for locating, classifying, and relating symbols. First, a Line-Of-Sight (LOS) graph is computed over the handwritten strokes in a formula. Second, class distributions for LOS nodes and edges are obtained using query-specific feature filters (i.e., attention) in a single feed-forward pass. Finally, a Maximum Spanning Tree (MST) is extracted from the weighted graph. Our preliminary results show that this is a promising new approach for visual parsing of handwritten formulas. Our data and source code are publicly available.