Bootstrapping Markov chains: countable case

Abstract

Let X be an irreducible, aperiodic and positive recurrent Markov chain with a countably infinite state space S and transition probability matrix P. Let π be the stationary probability and T_k be the first hitting time of a state k. Given a realization {xj;0≤j≤n} of {Xj;0≤j≤n}, let P_n be the maximum likelihood estimate of P. In this paper, the distribution of the naive bootstrap of the pivot √n( P_n-P) is shown to appropriate that of the pivot as n→∞. The approach used is via a double array of Markov chains for which a weak law and a central limit theorem are established. Next, in order to estimate analogous quantities for the stationary probability and hitting time distribution, two different bootstrap methods are discussed.