Supporting Efficient Fuzzy Queries on Large Text Repositories Using Hadoop

点击翻译按钮获取中文摘要

This project is studying research challenges to support efficient fuzzy queries on large text repositories using the MapReduce/Hadoop parallel computing paradigm. Supporting fuzzy queries is becoming increasingly more important in applications that need to deal with a variety of data inconsistencies in structures, representations, or semantics. Many existing algorithms require an offline analysis of data sets to construct an efficient index structure to support online query processing. Fuzzy join queries of data sets are more time consuming due to the computational complexity. The PI is studying three research problems: (1) constructing high-quality inverted lists for fuzzy search queries using Hadoop; (2) supporting fuzzy joins of large data sets using Hadoop; and (3) using the developed techniques to improve data quality of large collections of documents. The PI is collaborating with industrial partners on these topics.The techniques developed in this project will have a broad impact on many information systems that need to support approximate query processing on large data sets. The specific areas where the project is likely to have the most direct impact are Web search, enterprise search, data integration, data cleaning, and query relaxation. These areas have many data-intensive applications in scientific research, commercial systems, and Web-data management. The PI is also using the results of this research to provide teaching materials for students to learn the MapReduce/Hadoop computing paradigm to process large amounts of information.