A generalization of BIC for the general exponential family

Abstract

In a normal example of Stone (1979, J. Roy. Statist. Soc. Ser. B 41, 276-278), Berger et al. (2003, J. Statist. Plann. Inference 112, 241-258) showed BIC may be a poor approximation to the logarithm of Bayes Factor. They proposed a Generalized Bayes Information Criterion (GBIC) and a Laplace approximation to the log Bayes factor in that problem. We consider a fairly general case where one has p groups of observations coming from an arbitrary general exponential family with each group having a different parameter and r observations. We derive a GBIC and a Laplace approximation to the integrated likelihood, under the assumption that p→∞ and r→∞ (and some additional restrictions, which vary from example to example). The general derivation clarifies the structure of GBIC. A general theorem is presented to prove the accuracy of approximation, and the worst possible approximation error is derived for several examples. In several numerical examples, the Laplace approximation and GBIC are seen to be quite good. They perform much better than BIC.