Improving Information Retrieval with Machine Learning

包括各种搜索引擎在内的信息检索系统在我们的生活中扮演着重要的角色。它们包括通用网络搜索引擎(如Google)、垂直搜索引擎(如Amazon.com中的产品搜索)、特定媒体搜索(如图像搜索)等。在过去的几年里，我在机器学习的几个重要领域做出了卓有成效的重大贡献。这项研究计划的目的是扩大我在机器学习方面的研究，并利用它来改善信息检索系统的性能和用户体验，使人们能够轻松地获得高度准确和有用的信息。我们在这项研究计划中将研究的一个重要问题是如何利用层次结构来有效地提高搜索精度和用户体验。主题、流派、媒体类型等方面的层次结构或分类对人类来说是很自然的，在组织人类知识方面很重要。如果关键字搜索和分层浏览(向上滚动和向下钻取)可以无缝集成，那将是最理想的。例如，在艺术类别中搜索“中国”，就会消除中国作为国家的模棱两可。然而，在网络搜索引擎(如谷歌)中还没有做到这一点，因为数十亿个网页必须通过机器学习来分类(即，不是像DMOZ中的开放目录项目那样由人来分类)。其他引人入胜的研究挑战包括：使用泛化查询的主动学习(我们的工作)如何改善IR中的相关性反馈？我们如何才能利用许多“廉价”的标签器(如Amazon Machine Turk)来主动学习层级结构？我们如何学习信息检索系统用户的行为和认知模式？我们的研究计划的结果和影响将是富有成效和重大的。首先，它将极大地促进机器学习和信息检索研究。我们将继续发表收视率最高的会议和期刊论文。第二，我们将利用机器学习技术建立先进的信息检索系统和搜索引擎的原型。第三，我们将提高我们的IR原型在大规模用户研究中的可用性。我们这项研究的最终目标是让人们更容易地获得高度准确和有用的信息。

Information retrieval (IR) systems including various search engines have played important roles in our life. They include general web search engines (such as Google), vertical search engines (such as product search in Amazon.com), specific media search (such as image search), and so on. In the past years I have made fruitful and significant contributions in several important areas of machine learning. The objectives of this research program are to expand my research in machine learning, and leverage it to improve the performance and user experience of IR systems for people to easily obtain highly accurate and useful information.One important question we will study in this research program is how hierarchies can be used to effectively improve search accuracy and user experience. Hierarchies or taxonomies on topics, genres, media types, and so on, are natural to human and important in organizing human knowledge. It would be ideal if keyword search and hierarchical browsing (roll-up and drill-down) can be seamlessly integrated. For example, searching "china" in the art category will remove ambiguity of china as the country. However, this has not been done in web search engines (such as Google), because billions of webpages must be classified into hierarchies by machine learning (i.e., not by human as in DMOZ, the Open Directory Project). Other fascinating research challenges include: how can active learning with generalized queries (our work) improve relevance feedback in IR? How can we utilize many "cheap" labellers (such as Amazon Mechanical Turk) in active learning of hierarchies? How can we learn from behaviours and cognitive models of users of IR systems? The outcome and impact of our research program will be fruitful and significant. First of all, it will greatly advance both machine learning and IR research. We will continue to publish top-rated conference and journal papers. Second, we will build prototypes of advanced IR systems and search engines with machine learning techniques. Third, we will improve usability of our IR prototypes in large-scale user studies. Our ultimate objective of this research is to make it easy for people to obtain highly accurate and useful information.