Hierarchical ensemble of neural networks

Abstract

The estimation of the a posteriori probability p(c _k|x) given the state conditional probability distribution p(x|c_k) and a priori probability p(c_k) is the central theme in the Bayesian approach to the pattern classification problem. The a posteriori probability can be expressed in a product form p(g_m|x)p (c_k|xg_m). A classification scheme using a hierarchical ensemble of multilayer perceptrons (MLPs) is proposed based on this idea. This architecture is shown to be equivalent, in principle, to a single-stage MLP classifier. The advantages of the hierarchical ensemble of classifiers become apparent in practice where the probability estimates are computed from a finite set of samples in a finite time with a particular algorithm. With respect to given performance criteria, such as classification accuracy over a disjoint test set, a hierarchical ensemble performs better than an equivalent single-stage classifier, given a limited amount of resources in terms of input data and learning time. Experiments on vowel classification using a hierarchical scheme show these advantages over a single-stage classifier.