Fully Online Grammar Compression in Constant Space
Fully Online Grammar Compression in Constant Space
复制标题
恒定空间中的完全在线语法压缩
DOI：
10.1109/dcc.2014.69
复制
发表时间：
2014
期刊：
影响因子：
--
通讯作者：
Yasuo Tabei
中科院分区：
文献类型：
--
作者：
Shirou Maruyama;Yasuo Tabei
我们介绍了完全在线LCA（FOLCA）的新颖变体，这是一种完全在线语法压缩，构建了直线程序（SLP），并直接以在线方式将其编码为简洁的表示。 FOLCA可以将SLP直接编码为简洁的表示，该简洁表示等同于表示代表SLP的信息理论下限（Maruyama等人，Spire'13）。 FOLCA的压缩需要与输入文本的长度成正比的线性时间，其工作空间仅取决于SLP的大小，这使我们能够将FOLCA应用于大规模重复的文本。但是，最近的重复文本包括一些噪音。例如，当前的测序技术具有明显的错误率，将噪声嵌入基因组序列中。对于这种嘈杂的重复文本，在SLP尺寸的Folca会消耗大量内存。我们通过利用流挖掘技术背后的想法来展示在恒定空间中工作的两种变体。使用1000个人类基因组项目对应于约300GB的100种人类基因组的实验揭示了我们方法对大型，嘈杂的重复文本的适用性。
We present novel variants of fully online LCA (FOLCA), a fully online grammar compression that builds a straight line program (SLP) and directly encodes it into a succinct representation in an online manner. FOLCA enables a direct encoding of an SLP into a succinct representation that is asymptotically equivalent to an information theoretic lower bound for representing an SLP (Maruyama et al., SPIRE'13). The compression of FOLCA takes linear time proportional to the length of an input text and its working space depends only on the size of the SLP, which enables us to apply FOLCA to large-scale repetitive texts. Recent repetitive texts, however, include some noise. For example, current sequencing technology has significant error rates, which embeds noise into genome sequences. For such noisy repetitive texts, FOLCA working in the SLP size consumes a large amount of memory. We present two variants of FOLCA working in constant space by leveraging the idea behind stream mining techniques. Experiments using 100 human genomes corresponding to about 300GB from the 1000 human genomes project revealed the applicability of our method to large-scale, noisy repetitive texts.
影响因子：
1.8
作者：
Karp, RM;Shenker, S;Papadimitriou, CH
通讯作者：
Papadimitriou, CH