IMPROVING MIN HASH VIA THE CONTAINMENT INDEX WITH APPLICATIONS TO METAGENOMIC ANALYSIS
IMPROVING MIN HASH VIA THE CONTAINMENT INDEX WITH APPLICATIONS TO METAGENOMIC ANALYSIS
复制标题
通过包含索引改进最小哈希并应用于宏基因组分析
DOI：
10.1101/184150
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
H. Zabeti
中科院分区：
文献类型：
--
作者：
D. Koslicki;H. Zabeti
最小散列是一种概率方法，用于估计两个集合在Jaccard索引方面的相似性，Jaccard索引定义为它们的交集与并集的大小之比。我们证明了这种方法的性能最好的时候，所考虑的集是相似的大小和性能大大降低时，集是非常不同的大小。我们介绍了一种新的和有效的方法，称为遏制最小哈希方法，这是更适合于估计的Jaccard指数集的大小非常不同。我们通过利用另一种概率方法（特别是Bloom过滤器）来实现快速成员查询。我们推导出的包容最小哈希方法的估计错误的概率上的界限，并表明它显着提高了经典的最小哈希方法。我们还表现出显着的改进，在时间和空间的复杂性。作为应用，我们使用这种方法来检测宏基因组数据集中生物体的存在/不存在，表明它可以检测非常小的低丰度微生物的存在。
Min hash is a probabilistic method for estimating the similarity of two sets in terms of their Jaccard index, defined as the ration of the size of their intersection to their union. We demonstrate that this method performs best when the sets under consideration are of similar size and the performance degrades considerably when the sets are of very different size. We introduce a new and efficient approach, called the containment min hash approach, that is more suitable for estimating the Jaccard index of sets of very different size. We accomplish this by leveraging another probabilistic method (in particular, Bloom filters) for fast membership queries. We derive bounds on the probability of estimate errors for the containment min hash approach and show it significantly improves upon the classical min hash approach. We also show significant improvements in terms of time and space complexity. As an application, we use this method to detect the presence/absence of organisms in a metagenomic data set, showing that it can detect the presence of very small, low abundance microorganisms.