The Bernstein-Von Mises theorem for Markov processes

Abstract

Since the appearance of P. Billingsley's monograph [2] on the large sample inference in Markov processes in which the weak consistency and asymptotic normality of the maximum likelihood estimate was investigated, there has been considerable interest in the further development of the theory along other directions. Billingsley's work was mainly concerned with extending the results of H. Cramér ([4] page 500). Among more recent developments one might mention the proof of the almost sure consistency of the maximum likelihood estimator following the ideas of A. Wald by G. Roussas [7], and the results on asymptotic Bayes estimates obtained by Lorraine Schwartz [9]. In the present paper we extend to Markov processes one of the fundamental results in the asymptotic theory of inference, viz., the approach of the posterior density (in a sense to be made precise later) to the normal. When the observed chance variables are independent and identically distributed, this result was obtained by L. LeCam in [5] (page 309). The same author offers another derivation of this result in [6]. Special cases of the theorem were first given by S. Bernstein and R. von Mises (for reference see [5]). The regularity conditions satisfied by the transition probability density of the Markov process are given in Section 1. We prove in Theorem 2.4 of Section 2 those properties of the maximum likelihood estimator that are needed for the proof of the main result of the paper given in Section 3 (Theorem 3.1). Theorem 3.1 is stated in a form which is general enough to include the Bernstein-von Mises theorem as well as the somewhat sharper versions that are available when it is known that the prior probability distribution has a finite absolute moment of order m. Theorem 3.2 deduces these results as a consequence of Theorem 3.1. The latter result also enables us to prove a theorem on the asymptotic behavior of regular Bayes estimates. This is done in Theorem 4.1 of Section 4.