Deep Reinforcement Learning for Dialogue Systems

在几年的时间里，机器学习已经从一个由一小群专注的研究人员所追求的学术学科，变成了一个主流产业，准备好解决任何和所有可用数据的问题。这场革命的核心是一组灵活、健壮和强大的算法，其中许多基于神经网络和深度学习架构。部署深度学习技术的一个特别有前途的方向是构建语言理解和生成系统。******我们的工作旨在扩展深度学习方法，通过利用强化学习范式来解决自然语言理解和生成中的挑战性任务。***强化学习（RL）为解决未知（或部分已知）系统的顺序决策问题提供了丰富的数学和算法框架。******该项目主要侧重于为对话系统开发新的强化学习方法，特别是在对话管理系统中部署。选择这个领域是由于任务的顺序和交互性质，以及在不同时间点发生反馈的可能性，这与RL设置非常匹配。******在对话管理中，主要目标是建立一个可以使用自然语言与人类用户对话的人工智能代理。这样的系统可用于允许机器和用户之间进行不同类型的交互：社交聊天、获取信息（例如，客户服务）、进行交易（例如，网上银行或预订系统）。最终，一个完整的系统可以包含这些方面中的几个，从而允许丰富而频繁的交互模式。就我们的目标而言，对话管理任务是强化学习问题的一个经典例子，因为它要求机器学习一种良好的策略，以便在整个对话中产生一系列对话行为（响应）。******我们未来五年的目标是开发一系列适合在对话系统中部署的强化学习模型和算法，并提供具体的示例和性能质量的基准结果。我们还致力于在强化学习、深度学习和自然语言处理的交叉领域培养下一代具有研究生水平训练的机器学习研究人员。

In the space of a few years, machine learning has moved from an academic discipline pursued by a small group of dedicated researchers, to a mainstream industry, poised to solve any and all problems where data is available. At the core of this revolution are a set of flexible, robust and powerful algorithms, many of them based on neural network and deep learning architectures. One particularly promising direction for the deployment of deep learning technologies is in building language understanding and generation systems. ******Our work aims to extend deep learning methods to tackle challenging tasks in natural language understanding and generation by exploiting the reinforcement learning paradigm.***Reinforcement learning (RL) provides a rich mathematical and algorithmic framework for tackling the problem of sequential decision-making with an unknown (or partially known) system. ******This project focuses primarily on the development of new reinforcement learning methods for conversational systems, and in particular for deployment in Dialogue Management systems. This domain is chosen due to the sequential and interactive nature of the task, and the possibility for feedback to occur at different time points, which are a good match for the RL setting.******In Dialogue Management, the main goal is to build an AI agent that can converse with a human user using natural language. Such systems can be used to allow different types of interactions between a machine and user: social chatting, acquiring information (e.g., customer service), conducting transactions (e.g., online banking or reservation system). Eventually, a complete system can incorporate several of these aspects, allowing a rich and frequent pattern of interactions. In terms of our objectives, the Dialogue Management task is a classic example of an RL problem, as it requires the machine to learn a good strategy for producing a sequence of dialogue acts (responses) throughout the conversation.******Our goal over the next five years is to develop a range of RL models and algorithms suitable for deployment in dialogue systems, with specific examples and benchmark results of their performance quality. We also aim to develop the next generation of ML researchers with graduate-level training at the intersection of reinforcement learning, deep learning and NLP.