Introduction to HMM's

The fundamental assumption of an HMM is that the process to be modeled is governed by a finite number of states and that these states change once per time step in a random but statistically predictable way. To be more precise, let ${\rm Pr}(q_t = i)$ be the probability that the system transitions into state $i$ at time $t$. The Markovian assumption says that ${\rm Pr}(q_t = i)$ depends only on $q_{t-1}$, the true state at time $t-1$. Furthermore, if this distribution does not depend on the absolute time $t$, then the state probabilities can be described completely by a fixed state transition matrix ${\bf A} = \{a_{ij}\}$ where $a_{ij} = {\rm Pr} (q_t = j \vert q_{t-1} = i)$.

Figure 13.20: A hidden Markov model (HMM). As the state transitions occur from sample to sample, the observer, cannot see the states directly. Instead, the observer makes observations whose PDF depends on the state.

Figure 13.20 illustrates a hidden Markov model (HMM). At each time step (time running from left to right), the Markov model is in one of the five possible states. According to the Markovian assumption, the probability that the model is in state $j$ at time $t$ is governed only by the transition probability $a_{ij}$, where $i$ is the true state at time $t-1$. The Markov model is “hidden" from view by the observer who can only observe measurements ${\bf z}_t$ whose PDF is governed by the true state at each time step. The mathematics of the HMM are reviewed in section 13.3.2.