Forward-reverse EM algorithm for Markov chains

Abstract

We develop an EM algorithm for estimating parameters that determine the dynamics of a discrete time Markov chain evolving through a certain measurable state space. As a key tool for the construction of the EM method we develop forward-reverse representations for Markov chains conditioned on a certain terminal state. These representations may be considered as an extension of the earlier work [1] on conditional diffusions. We present several experiments and consider the convergence of the new EM algorithm.