III: Small: Modeling and Predicting Term Mismatch for Full-Text Retrieval

许多文本搜索引擎使用概率推理来确定一个词代表一个人有多好？S需要信息。术语出现在相关文档中的概率是多少？满足信息需求的文件？是概率信息检索理论中的一个基础量，然而以往的研究很少提供关于如何可靠地估计它的线索。该项目使用探索性数据分析来确定用户指定的查询条件与相关文档无法匹配的常见原因，开发与每个原因相关的功能，并将它们集成到一个可以从数据训练的模型中。术语必然性预测可用于最新的检索模型，以显著提高检索精度。术语必然性预测基于文本检索的两阶段方法。对初始检索的基于特征的分析产生了证据，这些证据可以与术语可能与相关文档不匹配的各种常见原因相关联，例如中心性、同义词和抽象性。这种基于模型的方法可以从现有数据中训练，从而很容易纳入测试新假设的新特征，或者训练特定于语料库的预测模型。它还有一个优势，即概率预测是特定于查询的，并且链接到可以指导自动术语加权以及交互或自动查询精化的功能。该项目为交互、自动查询扩展和查询的相关反馈改进开发了几种有针对性的干预措施。该项目通过为影响概率检索模型的中心问题提供新的方法，以及对查询形成中的问题进行诊断和纠正，对科学界产生了影响。搜索引擎精确度的提高也影响到了广大的日常用户。建议的研究提高了普通人的搜索准确率。使用非结构化关键字查询，以及经常使用复杂的结构化查询来搜索结构化文档的专业搜索者。研究结果将在研究论文中和通过项目网站(http://www.cs.cmu.edu/~callan/Projects/IIS-1018317/).传播新技术将在狐猴项目-S Indri搜索引擎(http://www.lemurproject.org/indri/).)的定期发布中实施和传播Indri被广泛的国际研究界使用，因此这种传播形式使其他研究人员更有可能研究和扩展拟议的研究。

Many text search engines use probabilistic reasoning to determine how well a word represents a person?s information need. The probability that a term appears in relevant documents ? documents that satisfy the information need ? is a fundamental quantity in the theory of probabilistic information retrieval, however prior research provided few clues about how to estimate it reliably. This project uses exploratory data analysis to identify common reasons that user-specified query terms fail to match relevant documents, develops features correlated with each reason, and integrates them into a model that can be trained from data. The resulting term necessity predictions can be used in state-of-the-art retrieval models to improve retrieval accuracy substantially.Term necessity predictions are based on a two-stage approach to text retrieval. A feature-based analysis of an initial retrieval develops evidence that can be linked to a variety of common reasons that a term might not match relevant documents, for example, centrality, synonymy, and abstractness. This model-based approach can be trained from available data, making it easy to incorporate new features that test new hypotheses, or to train a corpus-specific predictive model. It also has the advantage that probability predictions are query-specific, and linked to features that can guide automatic term weighting as well as interactive or automatic query refinement. The project develops several focused interventions for interactive, automatic query expansion, and relevance feedback refinement of queries.This project makes an impact on the scientific community by providing new approaches to a central problem that affects probabilistic retrieval models, and the diagnosis and correction of problems in query formation. Improvements in search engine accuracy also affect a broad population of everyday users. The proposed research improves search accuracy for ?ordinary people? using unstructured keyword queries, as well as professional searchers who often use sophisticated structured queries to search structured documents.Research results will be disseminated in research papers and via project web site (http://www.cs.cmu.edu/~callan/Projects/IIS-1018317/). New techniques will be implemented and disseminated in periodic releases of the Lemur Project?s Indri search engine (http://www.lemurproject.org/indri/). Indri is used by a broad international research community, thus this form of dissemination makes it more likely that other researchers will study and extend the proposed research.