Deep information retrieval - diving into information granularity and abstraction

虽然信息检索的最终目标是找到相关的(片段)信息，但信息检索的研究一般局限于检索整个文档。实际上，许多用户查询都是为了在文档中找到更精确的文本片段。我们在这项研究中处理的第一个问题是使最终用户能够找到不同粒度的答案。信息检索还使用唯一的文档表示。在实践中，用户的搜索意图差异很大：查询可能用于查找包含特定单词序列的文本，或包含所需语义的文本。唯一的表示不能满足这样的各种搜索意图。我们建议在不同的抽象层次上表示文本，从表层的单词到更抽象的语义表示。具有不同意图的查询可以与适当抽象级别的文本表示进行比较。最后，信息检索需要一个复杂的排名函数，这通常是从排名实例中学习的(学习到排名)。然而，学习到排名的方法从未与学习表示法结合使用。我们建议将所需的两项学习任务结合起来。此外，匹配函数还可能涉及使用领域知识来推断文档和查询之间的语义关系。上述问题将使用深度学习技术进行研究。文本的神经网络表示将在几个层上创建，对应于不同的抽象级别。此外，整个文档和其中的可搜索片段都被表示。学习排序方法将被训练以选择合适的方式来使用在网络中创建的表示来对文档进行排序。本研究的最终目标是开发能够理解和处理各种用户信息需求的更智能的信息检索系统。

Although the final goal of information retrieval (IR) is to find relevant (pieces of) information, the research in IR has been generally limited to retrieving entire documents. In practice, many user queries are intended to find more precise segments of texts within documents. The first problem we deal with in this research is to make it possible for end users to find answers of different levels of granularity.IR also uses a unique representation for a document. In practice, user's search intents vary greatly: a query may be used to find a text containing a specific sequence of words, or a text containing the required semantics. The unique representation is unable to satisfy such various search intents. We propose to represent texts at different levels of abstraction, from surface words to more abstract semantic representations. Queries with different intents could then be compared with text representations at appropriate levels of abstraction.Finally, IR needs a complex ranking function, which is usually learned from ranking examples (learning-to-rank). However, the learning-to-rank approaches have never been used in combination with learning representations. We propose to combine the two learning tasks that are required. In addition, the matching function may also involve the use of domain knowledge to infer the semantic relations between a document and a query.The above problems will be investigated using deep learning techniques. Neural network representations for texts will be created at several layers, corresponding to different levels of abstraction. In addition, both the entire document and the searchable segments in it are represented. The learning-to-rank method will be trained to select the appropriate way to use the representations created in the network to rank documents for a query.The ultimate goal of this research is to develop more intelligent IR systems that can understand and cope with various user information needs.