Bootstrapping a finite state Markov chain

Abstract

Consider a finite state discrete time ergodic Markov chain {X_n, n ≥0} with unknown transition matrix P and state space S = {1,..., m}. The matrix P can be estimated by P_n, the matrix of empirical transition rates. An estimator of the stationary distribution π or the distribution of the first hitting time of some state, say m, can be obtained by using P^{^}_n. However exact sampling distributions of the estimators are in general difficult to calculate. Here we use the boot strap technique to obtain approximation to these sampling distributions. It is shown that the boot-strap technique works in an asymptotic sense. Results are illustrated by some simulations.