Supporting Efficient Fuzzy Queries on Large Text Repositories Using Hadoop

该项目正在研究如何使用MapReduceHadoop并行计算范例在大型文本资源库上支持高效模糊查询的研究挑战。在需要处理结构、表示或语义中的各种数据不一致的应用中，支持模糊查询正变得越来越重要。现有的许多算法都需要对数据集进行离线分析，以构建高效的索引结构来支持在线查询处理。由于计算的复杂性，数据集的模糊连接查询更加耗时。PI正在研究三个研究问题：(1)使用Hadoop构建用于模糊搜索查询的高质量倒排列表；(2)使用Hadoop支持大数据集的模糊连接；(3)使用开发的技术来提高大文档集的数据质量。该项目开发的技术将对许多需要支持大数据集近似查询处理的信息系统产生广泛的影响。该项目可能产生最直接影响的特定领域是Web搜索、企业搜索、数据集成、数据清理和查询放松。这些领域在科学研究、商业系统和Web数据管理中有许多数据密集型应用程序。PI还利用这项研究的结果为学生提供教材，让他们学习MapReduce/Hadoop计算范例来处理大量信息。

This project is studying research challenges to support efficient fuzzy queries on large text repositories using the MapReduce/Hadoop parallel computing paradigm. Supporting fuzzy queries is becoming increasingly more important in applications that need to deal with a variety of data inconsistencies in structures, representations, or semantics. Many existing algorithms require an offline analysis of data sets to construct an efficient index structure to support online query processing. Fuzzy join queries of data sets are more time consuming due to the computational complexity. The PI is studying three research problems: (1) constructing high-quality inverted lists for fuzzy search queries using Hadoop; (2) supporting fuzzy joins of large data sets using Hadoop; and (3) using the developed techniques to improve data quality of large collections of documents. The PI is collaborating with industrial partners on these topics.The techniques developed in this project will have a broad impact on many information systems that need to support approximate query processing on large data sets. The specific areas where the project is likely to have the most direct impact are Web search, enterprise search, data integration, data cleaning, and query relaxation. These areas have many data-intensive applications in scientific research, commercial systems, and Web-data management. The PI is also using the results of this research to provide teaching materials for students to learn the MapReduce/Hadoop computing paradigm to process large amounts of information.