Search Engines Support for HLT Applications

点击翻译按钮获取中文摘要

This project extends search engine indexing and retrieval modelsdeveloped initially for structured (e.g., XML) documents to provideconvenient support for capabilities required by question answering,computer-assisted language learning, and other human languagetechnology (HLT) applications. These new capabilities are requiredbecause it is becoming common for HLT applications to use searchengines to find information in large text databases. HLT applicationstypically use extensive text annotations to reduce the mismatchbetween the word-based representations convenient for text retrievaland the concept-based representations convenient for reasoning. Theyalso can describe very specific requirements that retrieved textpassages must satisfy. This use of text search is very different thanad-hoc, interactive search, and current search engines do not supportit well. This project extends probabilistic indexing and retrievalmodels to support multiple, detailed, overlapping text annotations,hierarchical text annotations, annotations with associated confidencevalues, and similar capabilities. By explicitly recognizing HLTapplications as first-class users, the proposed research broadens theresearch community's view of text search well beyond the simplequeries, text representations, and retrieval models that characterizeinteractive, ad-hoc search today. It may also lead to improvedinteractive search by providing a medium in which search interfacescan express constraints and preferences derived from sophisticateduser and task models. Research results are disseminated in technicalpapers, and as part of the open-source Lemur Toolkit. The project Website (http://www.cs.cmu.edu/~callan/Projects/IIS-0534345/) providesadditional information.