Scalable and Reliable Conversational Search and Question Answering

点击翻译按钮获取中文摘要

People naturally ask questions and learn information through conversations. As conversational systems become more ubiquitous through their presence in messaging apps (e.g., Messenger and Wechat), video chat appliances (e.g, Echo), and personal digital assistants (e.g., Siri) they are beginning to supplant traditional web search (e.g., Google and Baidu) as day-to-day information sources. However, to a large extent, current conversational systems still focus on facilitating the completion of specific well understood tasks, such as playing music and e-commerce. Support for more general question answering and information seeking tasks remains limited. Straightforward adaptations of traditional web search engines for this purpose are restricted by the need to display large amounts of text and numerous web links, which is impractical on many of the interaction surfaces deploying conversational search, and by the keyword query and response paradigm of traditional web search, which is not conversational in nature. While conversational systems are complex, and no single research program can address all aspects, in this proposal I will tackle three core research problems: 1) How do we quantitatively measure the success of a conversational search and question answering systems in order to improve them? 2) How do we build systems that operate quickly enough to interact with people in a natural fashion? 3) How do we build systems that return answers that people can trust? To address these problems, I will extend my current research on compatibility measures and preference judgments with new methods to determine best answers by modeling and measuring people interacting with conversational systems. I will improve question answering efficiency both by developing methods for tuning multi-stage rankers and by improving first-stage retrieval methods, taking advantage of the opportunities afforded by modern neural rankers. I will extend ongoing research on misinformation detection to conversational search and question answering, and I will continue to foster work in the larger research community through benchmark evaluation efforts. This proposal provides substantial opportunities for student training, both through the creation of new research and the transfer of this research to industry. My work has always been of an applied nature, with the results validated through implementation, experimentation, and performance measurement. I expect research undertaken under this proposal to improve the technology people use to learn about the world. I expect students trained under this research program to work in Canadian and international companies as data scientists, research scientists, or software developers; or to become researchers and teachers in Canadian and international universities. Skills include experience with applied machine learning tools, search software, natural language processing tools, crowdsourcing methods, and related statistical methods.