Bayesian update of dialogue state: A POMDP framework for spoken dialogue systems
Bayesian update of dialogue state: A POMDP framework for spoken dialogue systems
复制标题
DOI：
10.1016/j.csl.2009.07.003
复制
发表时间：
2010-10-01
影响因子：
4.3
通讯作者：
Young, Steve
中科院分区：
文献类型：
--
作者：
Thomson, Blaise;Young, Steve
本文描述了一个统计动机的框架进行实时对话状态更新和政策学习的口语对话系统。该框架是基于部分可观察马尔可夫决策过程（POMDP），它提供了一个良好的基础上，口语对话管理的统计模型。然而，在POMDP模型中，精确的信念状态更新在计算上是难以处理的，因此必须使用近似方法。本文提出了一种基于循环置信传播算法的易处理方法。各种简化，提高效率显着相比，原始算法以及相比，其他POMDP为基础的对话状态更新方法。本文的第二个贡献是口语对话系统的学习方法，它使用了基于组件的政策与情节自然Actor Critic algorithm.The框架在本文中提出的模拟和用户试用进行了测试。两者都表明，使用贝叶斯更新的对话状态显着优于传统的定义的对话状态。政策学习有效地工作，学习的政策优于所有其他的模拟。在用户试验中，学习到的策略也具有竞争力，尽管其最优性不太确定。总体而言，贝叶斯更新的对话状态框架被证明是一个可行的和有效的方法来建立现实世界的POMDP为基础的对话系统。(C)2009爱思唯尔有限公司保留所有权利。
This paper describes a statistically motivated framework for performing real-time dialogue state updates and policy learning in a spoken dialogue system. The framework is based on the partially observable Markov decision process (POMDP), which provides a well-founded, statistical model of spoken dialogue management. However, exact belief state updates in a POMDP model are computationally intractable so approximate methods must be used. This paper presents a tractable method based on the loopy belief propagation algorithm. Various simplifications are made, which improve the efficiency significantly compared to the original algorithm as well as compared to other POMDP-based dialogue state updating approaches. A second contribution of this paper is a method for learning in spoken dialogue systems which uses a component-based policy with the episodic Natural Actor Critic algorithm.The framework proposed in this paper was tested on both simulations and in a user trial. Both indicated that using Bayesian updates of the dialogue state significantly outperforms traditional definitions of the dialogue state. Policy learning worked effectively and the learned policy outperformed all others on simulations. In user trials the learned policy was also competitive, although its optimality was less conclusive. Overall, the Bayesian update of dialogue state framework was shown to be a feasible and effective approach to building real-world POMDP-based dialogue systems. (C) 2009 Elsevier Ltd. All rights reserved.