Entropy for Hidden Markov Processes and Markov Random Fields

平稳马尔可夫过程作为随机现象的易处理模型是无处不在的。最简单的设置是 对一维随机序列建模的有限状态、离散时间平稳马尔可夫链。 这些过程是相当好理解的。在这个项目中，我们主要关注这一简单的 设置：1)平稳隐马尔可夫过程(HMP)和2)平稳马尔可夫随机场(MRF)。这个 前者模拟马尔可夫链产生的序列，并在存在噪声的情况下观察到。后者 在正方形、立方格子或其他类型的图形上对更高维度的随机数组进行建模 结构。对于从语音到语音的各种应用中的随机现象建模，两者都是不可或缺的 对网络通信的认识。对于这两类，我们都集中在有限状态、离散时间和 固定箱子。 平稳过程最基本的性质之一是它的熵，有时称为熵率。在设计用于数据压缩或纠错的代码时，计算熵是重要的第一步。虽然马尔可夫链的熵有一个简单的封闭形式的表达式，但很少有人能准确地计算出HMP或MRF的熵。硬方模型是一个经典的例子，几十年来，熵的计算一直难以解决。这个项目的主要目标是进一步发展HMP和MRF的熵的估计和渐近性，并解决相关的结构问题 对于这样的过程。我们将把以前关于HMP的工作从离散值过程扩展到连续值过程，并将改进我们先前在指数快速逼近熵方面的工作 用于MRF的。

Stationary Markov processes are ubiquitous as tractable models of random phenomena. The simplest setting is that of finite-state, discrete-time stationary Markov chains, which model one-dimensional random sequences. These processes are fairly well understood. In this project, we focus on two generalizations of this simple setting: 1) stationary hidden Markov processes (HMP) and 2) stationary Markov random fields (MRF). The former models sequences generated by a Markov chain and observed in the presence of noise. The latter models random arrays in higher dimensions, either on a square or cubic lattice or some other kind of graph structure. Both are indispensable for modeling random phenomena in applications ranging from speech recognition to network communications. For both classes, we focus on the finite-state, discrete-time and stationary cases. One of the the most fundamental properties of a stationary process is its entropy, sometimes called entropy rate. Computation of entropy is an important first step in the design of codes for data compression or error correction. While there is a simple, closed-form expression for the entropy of a Markov chain, it is rare that the entropy of an HMP or MRF can be computed exactly. The hard square model is a classical example, for which computation of entropy has eluded a solution for decades. The main goals of this project are to further develop estimates and asymptotics of entropy for HMP's and MRF's and to solve related structural problems for such processes. We will extend our previous work on HMP's from discrete-valued to continuous-valued processes and will improve our earlier work on exponentially-fast approximations to entropy for MRF's.