Deep Reinforcement Learning for Dialogue Systems

在短短几年的时间里，机器学习已经从一小群专注的研究人员追求的一门学术学科，发展成为一个主流行业，准备解决任何和所有可获得数据的问题。这场革命的核心是一套灵活、稳健和强大的算法，其中许多算法基于神经网络和深度学习架构。深度学习技术部署的一个特别有希望的方向是建立语言理解和生成系统。*我们的工作旨在扩展深度学习方法，通过利用强化学习范式来处理自然语言理解和生成中的挑战性任务。*强化学习(RL)为解决具有未知(或部分已知)系统的顺序决策问题提供了丰富的数学和算法框架。*本项目主要侧重于为对话系统开发新的强化学习方法，特别是在对话管理系统中的部署。选择此域是因为任务的顺序和交互性质，以及在不同时间点发生反馈的可能性，这与RL设置非常匹配。*在对话管理中，主要目标是构建一个可以使用自然语言与人类用户对话的人工智能代理。这样的系统可用于允许机器和用户之间的不同类型的交互：社交聊天、获取信息(例如，客户服务)、进行交易(例如，在线银行或预订系统)。最终，一个完整的系统可以结合这些方面中的几个，从而允许丰富而频繁的交互模式。就我们的目标而言，对话管理任务是RL问题的一个典型例子，因为它要求机器学习一种良好的策略，以便在整个对话过程中产生一系列对话行为(响应)。*我们的目标是在未来五年开发一系列适合在对话系统中部署的RL模型和算法，并提供具体的实例和它们的性能质量基准结果。我们还旨在培养下一代ML研究人员，在强化学习、深度学习和NLP的交叉点上进行研究生级别的培训。

In the space of a few years, machine learning has moved from an academic discipline pursued by a small group of dedicated researchers, to a mainstream industry, poised to solve any and all problems where data is available. At the core of this revolution are a set of flexible, robust and powerful algorithms, many of them based on neural network and deep learning architectures. One particularly promising direction for the deployment of deep learning technologies is in building language understanding and generation systems. ******Our work aims to extend deep learning methods to tackle challenging tasks in natural language understanding and generation by exploiting the reinforcement learning paradigm.***Reinforcement learning (RL) provides a rich mathematical and algorithmic framework for tackling the problem of sequential decision-making with an unknown (or partially known) system. ******This project focuses primarily on the development of new reinforcement learning methods for conversational systems, and in particular for deployment in Dialogue Management systems. This domain is chosen due to the sequential and interactive nature of the task, and the possibility for feedback to occur at different time points, which are a good match for the RL setting.******In Dialogue Management, the main goal is to build an AI agent that can converse with a human user using natural language. Such systems can be used to allow different types of interactions between a machine and user: social chatting, acquiring information (e.g., customer service), conducting transactions (e.g., online banking or reservation system). Eventually, a complete system can incorporate several of these aspects, allowing a rich and frequent pattern of interactions. In terms of our objectives, the Dialogue Management task is a classic example of an RL problem, as it requires the machine to learn a good strategy for producing a sequence of dialogue acts (responses) throughout the conversation.******Our goal over the next five years is to develop a range of RL models and algorithms suitable for deployment in dialogue systems, with specific examples and benchmark results of their performance quality. We also aim to develop the next generation of ML researchers with graduate-level training at the intersection of reinforcement learning, deep learning and NLP.