Supporting Efficient Fuzzy Queries on Large Text Repositories Using Hadoop

该项目正在研究使用MapReduce/Hadoop并行计算范式在大型文本存储库上支持高效模糊查询的研究挑战。 在需要处理结构、表示或语义上的各种数据不一致的应用程序中，支持模糊查询变得越来越重要。 许多现有的算法需要离线分析数据集，以构建一个有效的索引结构，以支持在线查询处理。数据集的模糊连接查询由于其计算复杂度而更加耗时。 PI正在研究三个研究问题：（1）使用Hadoop为模糊搜索查询构建高质量的倒排列表;（2）使用Hadoop支持大型数据集的模糊连接;以及（3）使用开发的技术来提高大型文档集合的数据质量。 PI正在与工业合作伙伴就这些主题进行合作。该项目中开发的技术将对许多需要支持大型数据集近似查询处理的信息系统产生广泛的影响。该项目可能产生最直接影响的具体领域是Web搜索、企业搜索、数据集成、数据清理和查询松弛。这些领域在科学研究、商业系统和Web数据管理中有许多数据密集型应用。 PI还利用这项研究的结果为学生提供教材，以学习MapReduce/Hadoop计算范式来处理大量信息。

This project is studying research challenges to support efficient fuzzy queries on large text repositories using the MapReduce/Hadoop parallel computing paradigm. Supporting fuzzy queries is becoming increasingly more important in applications that need to deal with a variety of data inconsistencies in structures, representations, or semantics. Many existing algorithms require an offline analysis of data sets to construct an efficient index structure to support online query processing. Fuzzy join queries of data sets are more time consuming due to the computational complexity. The PI is studying three research problems: (1) constructing high-quality inverted lists for fuzzy search queries using Hadoop; (2) supporting fuzzy joins of large data sets using Hadoop; and (3) using the developed techniques to improve data quality of large collections of documents. The PI is collaborating with industrial partners on these topics.The techniques developed in this project will have a broad impact on many information systems that need to support approximate query processing on large data sets. The specific areas where the project is likely to have the most direct impact are Web search, enterprise search, data integration, data cleaning, and query relaxation. These areas have many data-intensive applications in scientific research, commercial systems, and Web-data management. The PI is also using the results of this research to provide teaching materials for students to learn the MapReduce/Hadoop computing paradigm to process large amounts of information.