Spoken Dialogue Management using Partially Observable Markov Decision Processes

语音对话系统具有广泛的应用，包括呼叫中心自动化、家庭设备控制、交互式娱乐和免提应用。然而，尽管使用越来越多，部署费用仍然很高，业务系统仍然脆弱。这两个问题的一个主要贡献者是，核心的对话管理器，解释口头输入，并计划下一个响应是一个确定性的程序，手工制作和手动调整为每个application.Experience应用统计技术在语音识别和合成表明，从数据中学习，并使用最佳决策可以显着提高性能和降低成本。统计对话建模的一个自然框架是马尔可夫决策过程（MDP），然而，MDP的一个主要限制是它们需要准确地知道系统的状态，因此，它们没有解决对话管理问题的本质，即处理由语音识别和理解错误引起的不确定性。本项目的目的是开发一个口语对话系统的框架它使用一种更通用的统计模型，称为部分可观察马尔可夫决策过程（POMDP）。POMDP中的关键假设是系统的状态（包括用户心中的目标）永远无法确定。因此，它在所有可能的状态上保持一个概率分布，并根据这个分布做出决定。实际上，POMDP在每一个回合跟踪每一个可能的对话假设，为每一个保持一个概率。这为它提供了一个处理模糊性和不确定性的原则框架。虽然这个公式非常强大，但由于POMDP状态是一个非常高维连续空间中的向量，因此在计算上也非常复杂。这使得直接的信念监测和政策优化基本上是棘手的，因此几乎没有取得进展，走向真实的应用。然而，最近，提议者已经证明，实际POMDP为基础的系统是可行的，利用两个关键的想法。首先，通过将状态空间划分为等价类，大大降低了信任监控的复杂度。其次，在口语对话的背景下，可以将对话假设映射到一个大大减少的摘要空间中，从而可以进行有效的策略优化。这些想法已被内置到一个原型系统，称为隐藏信息状态（HIS）系统和他们的可行性已被证明和评估在旅游信息domain.Although它的目的作为一个概念的证明，HIS原型是使用一个简单的1-最好的识别器接口，非常简单的概率模型，手工制作的用户模拟器和一个基本的基于网格的策略学习方法。为了充分发挥基于POMDP的系统的潜力，还需要做更多的工作，本提案中提出的工作方案就是为了实现这一目标。将解决的关键领域是更有效的信念状态划分和监测，准确的统计用户模型训练的真实的数据，N-最好的识别假设的整合，并改善总结状态映射和政策优化。其结果将是一个系统，该系统在数据上进行自动训练，以低成本提供高性能，对识别错误具有更强的鲁棒性，并且可以在线学习和适应。

Spoken dialogue systems have a wide range of application including call centre automation, control of devices in the home, interactive entertainment, and hands-free applications. Despite their increasing use, however, deployment costs remain high and operational systems continue to be fragile. A major contributor to both of these problems is that the core dialogue manager which interprets the spoken input, and plans the next response is a deterministic program, hand-crafted and manually tuned for each application.Experience applying statistical techniques in both speech recognition and synthesis has shown that learning from data and using optimal decision making can dramatically improve performance and lower costs. A natural framework for statistical dialogue modelling is the Markov Decision Process (MDP), however, a major limitation of MDPs is that they require the state of the system to be known exactly, and therefore they do not address the essense of the dialogue management problem which is to handle the uncertainty caused by speech recognition and understanding errors.The aim of this project is to develop a framework for spoken dialogue systems which uses a more general statistical model called a Partially Observable Markov Decision Process (POMDP). The key assumption in the POMDP is that the state of the system (which includes the goal in the user's mind) can never be known with certainty. Hence, it maintains a probability distribution over all possible states and bases its decisions on this distribution. In effect, the POMDP tracks every possible dialogue hypothesis at every turn, maintaining a probability for each. This provides it with a principled framework for handling ambiguity and uncertainty.Although this formulation is extremely powerful, it is also computationally very complex since the POMDP state is a vector in a very high dimensional continuous space. This makes direct belief monitoring and policy optimisation essentially intractable and hence little progress has been made towards real applications. Recently, however, the proposer has demonstrated that practical POMDP-based systems are feasible by exploiting two key ideas. Firstly, the complexity of belief monitoring can be greatly reduced by partitioning the state space into equivalence classes. Secondly, in the context of spoken dialogues, it is possible to map dialogue hypotheses into a much-reduced summary space where effective policy optimisation is possible. These ideas have been built into a prototype system called the Hidden Information State (HIS) system and their feasibility has been demonstrated and evaluated in a Tourist Information domain.Although it serves its purpose as a proof of concept, the HIS prototype was built using a simple 1-best recogniser interface, very simplistic probabilistic models, a hand-crafted user simulator and a rudimentary grid-based policy learning method. To fully realise the potential of POMDP-based systems, much more needs to be done and the programme of work set out in this proposal seeks to achieve this. The key areas that will be addressed are more efficient belief state partitioning and monitoring, accurate statistical user models trained on real data, integration of N-best recognition hypotheses, and improved summary state mapping and policy optimisation. The result will be a system which is trained automatically on data, which delivers high performance at low cost, which is significantly more robust to recognition errors, and which can learn and adapt on-line.