Deep information retrieval - diving into information granularity and abstraction

虽然信息检索的最终目标是找到相关的信息，但信息检索的研究通常仅限于检索整个文档。在实践中，许多用户查询的目的是在文档中找到更精确的文本段。在这项研究中，我们处理的第一个问题是使最终用户能够找到不同粒度级别的答案。IR也使用了一个独特的表示文档。实际上，用户的搜索意图差异很大：查询可能用于查找包含特定单词序列的文本，或包含所需语义的文本。唯一表示无法满足这样的各种搜索意图。我们建议在不同的抽象层次上表示文本，从表面词到更抽象的语义表示。最后，IR需要一个复杂的排名功能，这通常是从排名示例（learning-to-rank）中学习的。然而，学习排名的方法从来没有与学习表示结合使用。我们建议将所需的两个学习任务联合收割机结合起来。此外，匹配功能还可能涉及使用领域知识来推断文档和查询之间的语义关系。上述问题将使用深度学习技术进行研究。文本的神经网络表示将在多个层次上创建，对应于不同的抽象层次。此外，整个文档和其中的可搜索段都被表示。学习排名的方法将被训练，以选择适当的方式来使用网络中创建的表示来排名文档的query.The最终目标是开发更智能的IR系统，可以理解和科普各种用户的信息需求。

Although the final goal of information retrieval (IR) is to find relevant (pieces of) information, the research in IR has been generally limited to retrieving entire documents. In practice, many user queries are intended to find more precise segments of texts within documents. The first problem we deal with in this research is to make it possible for end users to find answers of different levels of granularity.IR also uses a unique representation for a document. In practice, user's search intents vary greatly: a query may be used to find a text containing a specific sequence of words, or a text containing the required semantics. The unique representation is unable to satisfy such various search intents. We propose to represent texts at different levels of abstraction, from surface words to more abstract semantic representations. Queries with different intents could then be compared with text representations at appropriate levels of abstraction.Finally, IR needs a complex ranking function, which is usually learned from ranking examples (learning-to-rank). However, the learning-to-rank approaches have never been used in combination with learning representations. We propose to combine the two learning tasks that are required. In addition, the matching function may also involve the use of domain knowledge to infer the semantic relations between a document and a query.The above problems will be investigated using deep learning techniques. Neural network representations for texts will be created at several layers, corresponding to different levels of abstraction. In addition, both the entire document and the searchable segments in it are represented. The learning-to-rank method will be trained to select the appropriate way to use the representations created in the network to rank documents for a query.The ultimate goal of this research is to develop more intelligent IR systems that can understand and cope with various user information needs.