Entropy for Hidden Markov Processes and Markov Random Fields

隐马尔可夫过程和马尔可夫随机场的熵

• 批准号: 261611-2012
• 负责人: Marcus, Brian
• 金额: $ 3.06万
• 依托单位: University of British Columbia
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2015
• 资助国家: 加拿大
• 起止时间: 2015-01-01 至 2016-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=571097
• 关键词: Entropy; Hidden; Markov; Processes; Random

Stationary Markov processes are ubiquitous as tractable models of random phenomena. The simplest setting is that of finite-state, discrete-time stationary Markov chains, which model one-dimensional random sequences. These processes are fairly well understood. In this project, we focus on two generalizations of this simple setting: 1) stationary hidden Markov processes (HMP) and 2) stationary Markov random fields (MRF). The former models sequences generated by a Markov chain and observed in the presence of noise. The latter models random arrays in higher dimensions, either on a square or cubic lattice or some other kind of graph structure. Both are indispensable for modeling random phenomena in applications ranging from speech recognition to network communications. For both classes, we focus on the finite-state, discrete-time and stationary cases. One of the the most fundamental properties of a stationary process is its entropy, sometimes called entropy rate. Computation of entropy is an important first step in the design of codes for data compression or error correction. While there is a simple, closed-form expression for the entropy of a Markov chain, it is rare that the entropy of an HMP or MRF can be computed exactly. The hard square model is a classical example, for which computation of entropy has eluded a solution for decades. The main goals of this project are to further develop estimates and asymptotics of entropy for HMP's and MRF's and to solve related structural problems for such processes. We will extend our previous work on HMP's from discrete-valued to continuous-valued processes and will improve our earlier work on exponentially-fast approximations to entropy for MRF's.

平稳马尔可夫过程作为随机现象的易处理模型是普遍存在的。最简单的设置是 有限状态，离散时间平稳马尔可夫链，它模拟一维随机序列。 这些过程都是相当好理解的。在这个项目中，我们专注于这个简单的两个概括 设置：1）平稳隐马尔可夫过程（HMP）和2）平稳马尔可夫随机场（MRF）。的 前者模型序列产生的马尔可夫链和观察存在噪声。后者 在更高的维度上对随机阵列进行建模，无论是在正方形或立方格子上，还是在其他类型的图形上 结构这两个都是不可或缺的建模随机现象的应用，从语音 识别网络通信。对于这两个类，我们专注于有限状态，离散时间和 固定的箱子。 平稳过程最基本的性质之一是它的熵，有时称为熵率。熵的计算是设计数据压缩或纠错码的重要的第一步。虽然马尔可夫链的熵有一个简单的封闭式表达式，但HMP或MRF的熵很少能精确计算。硬平方模型是一个经典的例子，几十年来熵的计算一直没有解决。本计画的主要目标是进一步发展HMP和MRF的熵的估计和渐近性，并解决相关的结构问题 对于这样的过程。 我们将扩展我们以前的工作HMP的从离散值到连续值的过程，并将改善我们以前的工作指数快速近似熵 MRF的。