Spoken Dialogue Management using Partially Observable Markov Decision Processes

使用部分可观察马尔可夫决策过程的口语对话管理

• 批准号: EP/F013930/1
• 负责人: Stephen Young
• 金额: $ 45.96万
• 依托单位: University of Cambridge
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2007
• 资助国家: 英国
• 起止时间: 2007 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FF013930%2F1
• 关键词: Spoken; Dialogue; Management; using; Partially

Spoken dialogue systems have a wide range of application including call centre automation, control of devices in the home, interactive entertainment, and hands-free applications. Despite their increasing use, however, deployment costs remain high and operational systems continue to be fragile. A major contributor to both of these problems is that the core dialogue manager which interprets the spoken input, and plans the next response is a deterministic program, hand-crafted and manually tuned for each application.Experience applying statistical techniques in both speech recognition and synthesis has shown that learning from data and using optimal decision making can dramatically improve performance and lower costs. A natural framework for statistical dialogue modelling is the Markov Decision Process (MDP), however, a major limitation of MDPs is that they require the state of the system to be known exactly, and therefore they do not address the essense of the dialogue management problem which is to handle the uncertainty caused by speech recognition and understanding errors.The aim of this project is to develop a framework for spoken dialogue systems which uses a more general statistical model called a Partially Observable Markov Decision Process (POMDP). The key assumption in the POMDP is that the state of the system (which includes the goal in the user's mind) can never be known with certainty. Hence, it maintains a probability distribution over all possible states and bases its decisions on this distribution. In effect, the POMDP tracks every possible dialogue hypothesis at every turn, maintaining a probability for each. This provides it with a principled framework for handling ambiguity and uncertainty.Although this formulation is extremely powerful, it is also computationally very complex since the POMDP state is a vector in a very high dimensional continuous space. This makes direct belief monitoring and policy optimisation essentially intractable and hence little progress has been made towards real applications. Recently, however, the proposer has demonstrated that practical POMDP-based systems are feasible by exploiting two key ideas. Firstly, the complexity of belief monitoring can be greatly reduced by partitioning the state space into equivalence classes. Secondly, in the context of spoken dialogues, it is possible to map dialogue hypotheses into a much-reduced summary space where effective policy optimisation is possible. These ideas have been built into a prototype system called the Hidden Information State (HIS) system and their feasibility has been demonstrated and evaluated in a Tourist Information domain.Although it serves its purpose as a proof of concept, the HIS prototype was built using a simple 1-best recogniser interface, very simplistic probabilistic models, a hand-crafted user simulator and a rudimentary grid-based policy learning method. To fully realise the potential of POMDP-based systems, much more needs to be done and the programme of work set out in this proposal seeks to achieve this. The key areas that will be addressed are more efficient belief state partitioning and monitoring, accurate statistical user models trained on real data, integration of N-best recognition hypotheses, and improved summary state mapping and policy optimisation. The result will be a system which is trained automatically on data, which delivers high performance at low cost, which is significantly more robust to recognition errors, and which can learn and adapt on-line.

语音对话系统具有广泛的应用范围，包括呼叫中心自动化、家庭设备控制、互动娱乐和免提应用。然而，尽管使用越来越多，部署成本仍然很高，操作系统仍然很脆弱。这两个问题的主要原因是，解释语音输入并计划下一个响应的核心对话管理器是一个确定性程序，它是为每个应用程序手工编写和手动调优的。在语音识别和合成中应用统计技术的经验表明，从数据中学习和使用最佳决策可以显着提高性能并降低成本。统计对话建模的一个自然框架是马尔可夫决策过程（MDP），然而，MDP的一个主要限制是它们需要准确地知道系统的状态，因此它们没有解决对话管理问题的本质，即处理由语音识别和理解错误引起的不确定性。该项目的目的是为口语对话系统开发一个框架，该框架使用一种更通用的统计模型，称为部分可观察马尔可夫决策过程（POMDP）。POMDP中的关键假设是，系统的状态（其中包括用户心目中的目标）永远无法确定。因此，它在所有可能的状态上保持一个概率分布，并根据这个分布做出决策。实际上，POMDP在每个回合跟踪每个可能的对话假设，为每个假设保持一个概率。这为处理歧义和不确定性提供了原则性框架。虽然这个公式非常强大，但由于POMDP状态是一个非常高维连续空间中的向量，因此它在计算上也非常复杂。这使得直接的信念监控和策略优化基本上难以实现，因此在实际应用方面几乎没有取得进展。然而，最近，该提议者通过利用两个关键思想证明了实际的基于pomdp的系统是可行的。首先，通过将状态空间划分为等价类，大大降低了信念监控的复杂性。其次，在口语对话的背景下，有可能将对话假设映射到一个大大缩小的总结空间中，从而有可能进行有效的政策优化。这些想法已经被构建到一个被称为隐藏信息状态（HIS）系统的原型系统中，其可行性已经在旅游信息领域得到了证明和评估。虽然它的目的是作为概念证明，但HIS原型是使用简单的1-best识别器界面，非常简单的概率模型，手工制作的用户模拟器和基本的基于网格的策略学习方法构建的。为了充分发挥基于pomdp的系统的潜力，还需要做更多的工作，本建议中列出的工作方案旨在实现这一目标。将解决的关键领域是更有效的信念状态划分和监控，在真实数据上训练的准确统计用户模型，n -最佳识别假设的集成，以及改进的汇总状态映射和策略优化。结果将是一个在数据上自动训练的系统，它以低成本提供高性能，对识别错误具有更强的鲁棒性，并且可以在线学习和适应。

项目成果

Back-off action selection in summary space-based POMDP dialogue systems

基于空间的 POMDP 对话系统中的退避动作选择

• DOI: 10.1109/asru.2009.5373416
• 发表时间: 2009
• 作者: Gasic M

Parameter estimation for agenda-based user simulation

• 发表时间: 2010-09
• 作者: Simon Keizer;Milica Gasic;Filip Jurcícek;François Mairesse;Blaise Thomson;Kai Yu;S. Young

Effective handling of dialogue state in the hidden information state POMDP-based dialogue manager

隐藏信息状态下对话状态的有效处理 基于POMDP的对话管理器

• DOI: 10.1145/1966407.1966409
• 发表时间: 2011
• 期刊: ACM Transactions on Speech and Language Processing
• 作者: Gašic M

The Hidden Agenda User Simulation Model

• DOI: 10.1109/tasl.2008.2012071
• 发表时间: 2009-05
• 期刊: IEEE Transactions on Audio, Speech, and Language Processing
• 作者: J. Schatzmann;S. Young

Phrase-Based Statistical Language Generation Using Graphical Models and Active Learning

• 发表时间: 2010-07
• 作者: François Mairesse;Milica Gasic;Filip Jurcícek;Simon Keizer;Blaise Thomson;Kai Yu;S. Young