Markov chain analysis