Improving Information Retrieval with Machine Learning

包括各种搜索引擎在内的信息检索系统在我们的生活中扮演着重要的角色。它们包括一般的web搜索引擎（如b谷歌）、垂直搜索引擎（如Amazon.com中的产品搜索）、特定媒体搜索（如图像搜索）等等。在过去的几年里，我在机器学习的几个重要领域做出了卓有成效的重大贡献。这个研究项目的目标是扩展我在机器学习方面的研究，并利用它来提高IR系统的性能和用户体验，让人们轻松获得高度准确和有用的信息。我们将在这个研究项目中研究的一个重要问题是如何使用层次结构来有效地提高搜索准确性和用户体验。关于主题、类型、媒体类型等的层次结构或分类法对人类来说是很自然的，并且在组织人类知识方面很重要。如果关键字搜索和分层浏览（上卷和下钻）能够无缝集成，那将是最理想的。例如，在艺术类别中搜索“中国”，将消除中国作为国家的模糊性。然而，这并没有在网络搜索引擎（如b谷歌）中完成，因为数十亿的网页必须通过机器学习（即，不是像DMOZ，开放目录项目那样由人类）分类成层次结构。其他有趣的研究挑战包括：如何使用广义查询（我们的工作）主动学习来改善IR中的相关性反馈？我们如何利用许多“廉价”的标签器（如Amazon Mechanical Turk）来主动学习层次结构？我们如何从IR系统用户的行为和认知模型中学习？我们的研究计划的结果和影响将是富有成效和显著的。首先，它将极大地推动机器学习和IR研究。我们将继续发表高水平的会议和期刊论文。其次，我们将用机器学习技术构建先进的红外系统和搜索引擎的原型。第三，我们将在大规模用户研究中提高IR原型的可用性。我们这项研究的最终目的是使人们更容易获得高度准确和有用的信息。

Information retrieval (IR) systems including various search engines have played important roles in our life. They include general web search engines (such as Google), vertical search engines (such as product search in Amazon.com), specific media search (such as image search), and so on. In the past years I have made fruitful and significant contributions in several important areas of machine learning. The objectives of this research program are to expand my research in machine learning, and leverage it to improve the performance and user experience of IR systems for people to easily obtain highly accurate and useful information.One important question we will study in this research program is how hierarchies can be used to effectively improve search accuracy and user experience. Hierarchies or taxonomies on topics, genres, media types, and so on, are natural to human and important in organizing human knowledge. It would be ideal if keyword search and hierarchical browsing (roll-up and drill-down) can be seamlessly integrated. For example, searching "china" in the art category will remove ambiguity of china as the country. However, this has not been done in web search engines (such as Google), because billions of webpages must be classified into hierarchies by machine learning (i.e., not by human as in DMOZ, the Open Directory Project). Other fascinating research challenges include: how can active learning with generalized queries (our work) improve relevance feedback in IR? How can we utilize many "cheap" labellers (such as Amazon Mechanical Turk) in active learning of hierarchies? How can we learn from behaviours and cognitive models of users of IR systems? The outcome and impact of our research program will be fruitful and significant. First of all, it will greatly advance both machine learning and IR research. We will continue to publish top-rated conference and journal papers. Second, we will build prototypes of advanced IR systems and search engines with machine learning techniques. Third, we will improve usability of our IR prototypes in large-scale user studies. Our ultimate objective of this research is to make it easy for people to obtain highly accurate and useful information.