Scalable and Reliable Conversational Search and Question Answering

人们自然会问问题，并通过对话了解信息。随着会话系统通过它们在消息应用中的存在而变得更加普遍（例如，Messenger和微信）、视频聊天设备（例如，Echo）和个人数字助理（例如，Siri），它们开始取代传统的网络搜索（例如，谷歌和百度）作为日常信息来源。然而，在很大程度上，当前的会话系统仍然专注于促进完成特定的良好理解的任务，例如播放音乐和电子商务。对更一般的问题回答和信息搜索任务的支持仍然有限。传统网络搜索引擎为此目的的直接适应受到显示大量文本和众多网络链接的需要的限制，这在部署对话式搜索的许多交互表面上是不切实际的，并且受到传统网络搜索的关键字查询和响应范例的限制，这在本质上不是对话式的。虽然会话系统是复杂的，没有一个单一的研究计划可以解决所有方面，在这个建议中，我将解决三个核心研究问题：1）我们如何定量衡量会话搜索和问答系统的成功，以改善他们？2)我们如何构建一个运行速度足够快的系统，以自然的方式与人们互动？3)我们如何建立一个系统，让人们可以信任的答案？为了解决这些问题，我将扩展我目前的研究兼容性措施和偏好判断的新方法，以确定最佳答案，通过建模和测量人与会话系统的互动。我将通过开发调整多阶段排序器的方法和改进第一阶段检索方法来提高问题回答效率，并利用现代神经排序器提供的机会。我将把正在进行的错误信息检测研究扩展到会话搜索和问答，我将继续通过基准评估工作促进更大研究社区的工作。该提案为学生培训提供了大量机会，无论是通过创造新的研究，还是将这项研究转移到工业领域。我的工作一直是应用性质的，通过实施，实验和性能测量验证的结果。我希望根据这一建议进行的研究能够改善人们用来了解世界的技术。我希望在这个研究项目下培训的学生在加拿大和国际公司工作，作为数据科学家，研究科学家或软件开发人员;或者成为加拿大和国际大学的研究人员和教师。技能包括应用机器学习工具，搜索软件，自然语言处理工具，众包方法和相关统计方法的经验。

People naturally ask questions and learn information through conversations. As conversational systems become more ubiquitous through their presence in messaging apps (e.g., Messenger and Wechat), video chat appliances (e.g, Echo), and personal digital assistants (e.g., Siri) they are beginning to supplant traditional web search (e.g., Google and Baidu) as day-to-day information sources. However, to a large extent, current conversational systems still focus on facilitating the completion of specific well understood tasks, such as playing music and e-commerce. Support for more general question answering and information seeking tasks remains limited. Straightforward adaptations of traditional web search engines for this purpose are restricted by the need to display large amounts of text and numerous web links, which is impractical on many of the interaction surfaces deploying conversational search, and by the keyword query and response paradigm of traditional web search, which is not conversational in nature. While conversational systems are complex, and no single research program can address all aspects, in this proposal I will tackle three core research problems: 1) How do we quantitatively measure the success of a conversational search and question answering systems in order to improve them? 2) How do we build systems that operate quickly enough to interact with people in a natural fashion? 3) How do we build systems that return answers that people can trust? To address these problems, I will extend my current research on compatibility measures and preference judgments with new methods to determine best answers by modeling and measuring people interacting with conversational systems. I will improve question answering efficiency both by developing methods for tuning multi-stage rankers and by improving first-stage retrieval methods, taking advantage of the opportunities afforded by modern neural rankers. I will extend ongoing research on misinformation detection to conversational search and question answering, and I will continue to foster work in the larger research community through benchmark evaluation efforts. This proposal provides substantial opportunities for student training, both through the creation of new research and the transfer of this research to industry. My work has always been of an applied nature, with the results validated through implementation, experimentation, and performance measurement. I expect research undertaken under this proposal to improve the technology people use to learn about the world. I expect students trained under this research program to work in Canadian and international companies as data scientists, research scientists, or software developers; or to become researchers and teachers in Canadian and international universities. Skills include experience with applied machine learning tools, search software, natural language processing tools, crowdsourcing methods, and related statistical methods.