Hidden semi Markov models; stochastic control.

点击翻译按钮获取中文摘要

In addition to signal processing, speech processing and other areas, one of the most successful applications of hidden Markov models has been to genome and protein sequencing. My book with L Aggoun and J Moore on "Hidden Markov Models: Estimation and Control" was published in 1995 by Springer Verlag and reprinted in 1997. A revised and expanded edition was printed in 2008. I have published several other contributions to hidden Markov models. However, when modelling discrete sequences, for example in discrete time or in biological applications, if the hidden process is a Markov chain its (random) occupation time in any state is a geometrically distributed random variable. For biological applications it appears that more general occupation times should be considered. This leads us to consider semi-Markov models where the probability of jumping to a new state depends not only on the present state but how long the chain has been in that state. We shall consider 'hidden' semi-Markov models, that is situations where the semi-Markov chain is not observed directly but modulates a second, observed, process. We believe these will better model behaviour exhibited in protein and other 'gnome' sequencing. We shall develop for these models the results found in our book and previous works. A second line of research will use our recent results on backward stochastic differential equations to investigate partially observed stochastic control problems. The adjoint process is described by a backward stochastic differential equation; for partially observed problems this is a backward stochastic partial differential equation. Initial work will consider this problem for the control of a partially observed Markov chain